Lori Baumgartner

Posted on Aug 22, 2019

Rails 5.2.x --> Rails 6 counter_cache Gotcha

#rails #ruby #migration #upgrade

I recently upgraded our production app from Rails 5.2.3 to Rails 6.0.0. We wanted access to all the shiny things everyone is talking about. Multiple databases! Better autoloading!

Everything was going great until I noticed one weird behavior where a count column wasn't getting updated, but all the associations were correctly updating. After talking with a coworker about the problem, we identified what was going on. The culprit? Rails ActiveRecord counter_cache.

This post will give a brief overview of counter_cache and how its behavior changed between the two Rails versions, and finally, how to update your code to get around this issue without throwing counter_cache out the window.

Counter Caching - What is it? Why would I use it?

This concept was new-to-me - which is great! I love learning new things and working towards understanding why this approach is better over others. In this case, it was older code and we couldn't ask the original writer why they made this decision. So off to the internet I went.

I found quite a few articles about how to implement counter_cache but was surprised to see that none of them were from official Rails documentation - which was interesting.

That said, I think the best resource I found was this (very old) Railscast episode about converting fetching the count on a record and its relations to counter caching in the database. I highly recommend watching it - but here's the short answer of why you would choose to implement counter caching:

A counter cache column stores the count of its associations on itself - which means you don't have to make extra database calls = more performant

Old Behavior & Implementation

Let's say we have a cookie jar and each cookie jar has many cookies inside it. Somewhere in our UI, we want to show how many cookies are in each cookie jar.

A simple implementation of setting up our models might look like this:

# cookie_jar.rb
class CookieJar
  has_many :cookies
end

# ## Schema Information
# Table name: `cookie_jars`
# ### Columns
#
# Name                 | Type               | Attributes
# -------------------- | ------------------ | ---------------
# **`id`**             | `bigint(8)`        | `not null, primary key`
# **`cookies_count`**  | `integer`          | `default(0), not null`

# cookie.rb
class Cookie
  belongs_to :cookie_jar, counter_cache: true
end

# ## Schema Information
# Table name: `cookies`
# ### Columns
#
# Name                 | Type               | Attributes
# -------------------- | ------------------ | ---------------
# **`id`**             | `bigint(8)`        | `not null, primary key`
# **`cookie_jar_id`**  | `integer`          | `not null`

So that means that we now have CookieJar, Cookie and the cookie_jars table has a column called cookies_count that uses counter_cache to calculate those values.

So now the tricky part - counter_cache is doing some work behind the scenes, which makes it easy to use, but more challenging to debug. In this case, our bug source was in some seed code that looked like this:

# seed_cookie_jar.rb
def assign_cookies
  jar = CookieJar.first
  jar.cookies.each do |cookie|
    cookie.update!(cookie_jar_id: jar.id)
  end
end

Just a note that in this example, let's not worry about where our cookies were created - just that they previously had a different cookie_jar_id that we want to change here in assign_cookies.

So, before Rails 6, this behavior worked - I move my first cookie from one jar to another and the old jar now has one less cookies_count and the new jar has incremented its cookies_count.

Our Cookie Jar table before we call assign_cookies

id	cookies_count
1	4
2	0

What our Cookie Jar table looks like after we move the cookies around

id	cookies_count
1	0
2	4

Everything is working as expected! All 4 of our cookies moved from Cookie Jar 1 to Cookie Jar 2. Delightful. Now let's wreck havoc on this and upgrade to Rails 6 💪

New Behavior & Changes Made

To be fair, there was a one-line reference to the pull request that implemented the new behavior. However, what surprised me was that the way the change was phrased in the changelog did not at all alert me that my code was now essentially broken.

Here's the changelog note: "Don't update counter cache unless the record is actually saved". Yup, that's it. Now, I'm not a Ruby or Rails pro, so maybe this was obvious behavior to other people...but boy-o it took me for a ride!

Without doing anything to our code, here was the bug we had:
Our Cookie Jar table before we call assign_cookies (same as last time)

id	cookies_count
1	4
2	0

What our Cookie Jar table looks like after we invoke assign_cookies - no errors, but those counts don't look right 🤔

id	cookies_count
1	4
2	0

More confusingly, what our Cookies table looks like after we move the cookies:

id	cookie_jar_id
1	2
2	2
3	2
4	2

WHAT?! Our cookies actually moved jars, but the counter didn't update. So in our UI, we're displaying that Cookie Jar 1 has 4 cookies in it, but when you click to see the detail of that Cookie Jar 1 - it's empty!

To be honest, the way my coworker and I figured this out was going to the pull request listed in the changelog as changing this code and looking at the code diffs, new tests cases, and try some stuff out. All that eventually led to us trying one thing: instead of passing an id value, pass the whole record to the block. (Spoiler alert - this worked!) Here's that code:

# seed_cookie_jar.rb
def assign_cookies
  jar = CookieJar.first
  jar.cookies.each do |cookie|
    cookie.update!(cookie_jar: jar)
  end
end

So no more cookie_jar_id set directly - let's pass it the whole jar instance and see what happens. Turns out, this does call a save on the jar instance and triggers the cache to update.

Thanks for sticking with me this far - hope you learned something new and are more cautious about updating association _id fields now! I definitely will think twice before passing in an explicit _id value vs. the instance from now on.

Photo by Brooke Lark on Unsplash

Top comments (6)

Daniel • Aug 23 '19

Thanks for the insight!

Incidentally, apart from this precise case, I have never been able to decide when it is better to pass ActiveRecord an id, as in .where shelf_id: shelf.id, or the entire object .where shelf: shelf.

The second example seems more idiomatic, but I never knew when to favor one or the other.

By the way, I used the method where as an example, but it could be any other method that accepts an ActiveRecord instance or an id.

asciigeek • Aug 23 '19

In most other ORM’s, in other frameworks, it is common to change those relationships using the entire object and not just setting the foreign key relationship with a primary key of a parent object. Clearly, rails has decided to go down that same avenue and it makes sense.

A proper framework should result in identically performing SQL whether you were specific in .where shelf_id: = shelf.id

.where shelf:=shelf

In the second case, the framework would understand that the primary key of shelf is ‘id’ and only utilize that one column in the query.

Lud • Aug 23 '19

In an object oriented world, I believe we should pass objects around, not properties of objects. I guess that's what you mean with "idiomatic" already. Of course there are exceptions, e.g. when passing it to an asynchronously called worker, to avoid problems when dumping the payload into Redis as a json value, but usually you instantiate the object in said worker in the first line, to continue working with, well, objects.

Besides this, thank you for sharing your experience in this article, you're for sure not the only one running into this problem :-)

Lori Baumgartner • Aug 23 '19

Totally agree! It's the one mystery of this change that I really want to investigate - because it's just not clear to me which to use when - and you're right, it's a decision we have to make in many other instances.