Photo by Paweł Czerwiński on Unsplash
In all the time that I've been building and maintaining Rails apps, I've somehow never noticed that ActiveRecord attributes backed by columns with default values are nil on create.
I'm going to demonstrate a number of ways to avoid this bit of confusion, but first let's start with an example.
Magic Links and Missing Tokens
This is a real example from a recent Rails side project. I added a MagicLink model and magic_links table for implementing magic sign in links for authentication. A magic link is a short-lived link that gets emailed to a user which when clicked will sign the user into the app.
Here is the schema description for the magic_links table in PostgreSQL.
\d magic_links
Table "public.magic_links"
Column | Type | Collation | Nullable | Default
-------------+--------------------------------+-----------+----------+------------------------------------
id | uuid | | not null | gen_random_uuid()
user_id | uuid | | |
token | uuid | | not null | gen_random_uuid()
expires_at | timestamp without time zone | | not null |
created_at | timestamp(6) without time zone | | not null |
updated_at | timestamp(6) without time zone | | not null |
If you scroll to the right on that code block, you'll see that both id and token have default values of gen_random_uuid(). This is a function in Postgres that can produce a random (v4) UUID -- as a random, unique identifier it makes a great id and a great magic sign in token.
Here is the accompanying ActiveRecord model, though not a lot to see.
class MagicLink < ApplicationRecord
belongs_to :user
end
And lastly, here is what I get when I create a new MagicLink record.
> magic_link = MagicLink.create(expires_at: 5.minutes.from_now, user: User.last)
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1 [["LIMIT", 1]]
TRANSACTION (0.1ms) BEGIN
MagicLink Create (0.4ms) INSERT INTO "magic_links" ("user_id", "expires_at", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["user_id", "f683477e-c284-4677-8804-3852fa82b799"], ["expires_at", "2021-01-19 00:54:01.490924"], ["created_at", "2021-01-19 00:49:01.492309"], ["updated_at", "2021-01-19 00:49:01.492309"]]
TRANSACTION (0.2ms) COMMIT
=> #<MagicLink:0x00007fcd0691eff8
id: "9fe2c468-fa11-4df3-9683-d11eb45aff06",
user_id: "f683477e-c284-4677-8804-3852fa82b799",
token: nil,
expires_at: Tue, 19 Jan 2021 00:54:01.490924000 UTC +00:00,
created_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
updated_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
used_at: nil>
The #create call inserts a new record into the magic_links table and produces an in-memory representation of that record which is an instance of MagicLink. There are two things I want to note about the in-memory (ActiveRecord) object.
The
idhas been set to"9fe2c468-fa11-4df3-9683-d11eb45aff06". Notice in theinsertstatement right about it that no value was sent to the Postgres server forid. Because of that, Postgres uses the default value. It invokesgen_random_uuid()to produce the record'sid.The
tokenvalue appears to benil. No value was passed fortokenas part of theinsertstatement. Because of that, I can assure you that Postgres invokedgen_random_uuid()for this field as well.
So, why is token showing up as nil in that MagicLink object?
To answer that, let's look a little closer at the insert statement that was produced by the #create call.
INSERT INTO "magic_links"
("user_id", "expires_at", "created_at", "updated_at")
VALUES ($1, $2, $3, $4)
RETURNING "id"
[...]
That statement has a returning clause which specifies exactly "id" and nothing else. The result of ActiveRecord running that query will be:
- Inserting a new record into
magic_linkswith the provided values and any necessary defaults. - Returning just the newly produced
idvalue for that record and nothing else.
When ActiveRecord gets back the response from the Postgres server for that insert, it will take the new information (just the id) and combine that with the info already embedded in the MagicLink object. The token value was not returned by the insert statement, so ActiveRecord doesn't know what it is. That's why token is nil.
So, what can we do about this?
A Variety of Solutions
There are several ways of getting at these default values. They all have tradeoffs, so you'll want to weigh those against your particular scenario.
1. Reload
Perhaps the first and most straightforward solution is to reload the record.
> magic_link.reload
MagicLink Load (0.4ms) SELECT "magic_links".* FROM "magic_links" WHERE "magic_links"."id" = $1 LIMIT $2 [["id", "9fe2c468-fa11-4df3-9683-d11eb45aff06"], ["LIMIT", 1]]
=> #<MagicLink:0x00007fcd0691eff8
id: "9fe2c468-fa11-4df3-9683-d11eb45aff06",
user_id: "f683477e-c284-4677-8804-3852fa82b799",
token: "b90427cb-ab54-4e08-97dd-f8e02aa2820c",
expires_at: Tue, 19 Jan 2021 00:54:01.490924000 UTC +00:00,
created_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
updated_at: Tue, 19 Jan 2021 00:49:01.492309000 UTC +00:00,
used_at: nil>
This will make a (wasteful?) roundtrip to the database, in addition to the initial insert to fetch all the up-to-date version of this object, all defaults included.
2. Produce the Default Value in Rails-Land
Most default values can be produced on the Rails side of things at the time of the create call.
> magic_link = MagicLink.create(
expires_at: 5.minutes.from_now,
user: User.last,
token: SecureRandom.uuid
)
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1 [["LIMIT", 1]]
TRANSACTION (0.1ms) BEGIN
MagicLink Create (0.4ms) INSERT INTO "magic_links" ("user_id", "token", "expires_at", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["user_id", "f683477e-c284-4677-8804-3852fa82b799"], ["token", "33fc275e-4235-4c59-9e8c-69c5ba622a3c"], ["expires_at", "2021-01-19 01:16:09.026531"], ["created_at", "2021-01-19 01:11:09.029372"], ["updated_at", "2021-01-19 01:11:09.029372"]]
TRANSACTION (0.3ms) COMMIT
=> #<MagicLink:0x00007fcd069aeba8
id: "43c4956c-87f0-40f3-933c-9a382057d464",
user_id: "f683477e-c284-4677-8804-3852fa82b799",
token: "33fc275e-4235-4c59-9e8c-69c5ba622a3c",
expires_at: Tue, 19 Jan 2021 01:16:09.026531000 UTC +00:00,
created_at: Tue, 19 Jan 2021 01:11:09.029372000 UTC +00:00,
updated_at: Tue, 19 Jan 2021 01:11:09.029372000 UTC +00:00,
used_at: nil>
This approach gets you back to a single round-trip with the database.
I find this solution irksome because I want to be able to rely on the database for things. Postgres provides many conveniences and affordances including the assurance of default values. I want to avoid duplicating that logic in my Rails code as much as possible.
That said, based on this particular magic link use case and my code maintenance preferences, this is my preferred solution.
3. Insert with overridden returning
ActiveRecord's #insert accepts an optional returning argument that allows you to override the default value of the returning clause. Setting it as returning: '*' adjust the insert SQL statement to return everything in the result, not just the id.
> result = MagicLink.insert(
{
expires_at: 5.minutes.from_now,
user_id: User.last.id,
created_at: Time.zone.now,
updated_at: Time.zone.now
}, returning: "*")
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1 [["LIMIT", 1]]
MagicLink Insert (0.8ms) INSERT INTO "magic_links" ("expires_at","user_id","created_at","updated_at") VALUES ('2021-01-19 01:20:00.912387', 'f683477e-c284-4677-8804-3852fa82b799', '2021-01-19 01:15:00.913344', '2021-01-19 01:15:00.913351') ON CONFLICT DO NOTHING RETURNING *
=> #<ActiveRecord::Result:0x00007fcd1722dbe8 ... >
> result.rows.first[result.columns.index("token")]
=> "92f64a62-8136-4243-9da9-f91c5786255f"
This is another single roundtrip solution. The drawbacks are:
The result of the call is an
ActiveRecord::Resultobject instead of aMagicLinkobject.The
#insertcall skips a bunch of the utilities of the#createcall including validations and callbacks. Notice also that I had to manually specify thecreated_atandupdated_atvalues.
4. Improved insert with overridden returning
The first drawback of the previous approach was that you get back an ActiveRecord::Result object instead of a MagicLink object. This can be solved by splatting and wrapping the whole thing in a MagicLink.new call.
> MagicLink.new(
**MagicLink.insert(
{
expires_at: 5.minutes.from_now,
user_id: User.last.id,
created_at: Time.zone.now,
updated_at: Time.zone.now
}, returning: "*"
).first)
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" DESC LIMIT $1 [["LIMIT", 1]]
MagicLink Insert (0.9ms) INSERT INTO "magic_links" ("expires_at","user_id","created_at","updated_at") VALUES ('2021-01-19 01:33:27.064213', 'f683477e-c284-4677-8804-3852fa82b799', '2021-01-19 01:28:27.065227', '2021-01-19 01:28:27.065234') ON CONFLICT DO NOTHING RETURNING *
=> #<MagicLink:0x00007fcd06a25dc0
id: "bb17ad85-7e19-4cbf-9821-21f9c631243b",
user_id: "f683477e-c284-4677-8804-3852fa82b799",
token: "4e99b551-60a2-4956-aba9-394ac2a1c9f0",
expires_at: Tue, 19 Jan 2021 01:33:27.064213000 UTC +00:00,
created_at: Tue, 19 Jan 2021 01:28:27.065227000 UTC +00:00,
updated_at: Tue, 19 Jan 2021 01:28:27.065234000 UTC +00:00,
used_at: nil>
This still suffers from missing out on callbacks and validations and having to specify timestamps.
5. Improved+ insert with overridden returning
At this point I feel like we are starting to jump the shark. This solution adds a custom #create_returning method that simulates a #create call that gets to include a returning override. It create a new object of the model's type with the given attributes, tries the #insert call from previous solutions if the attributes are valid, and then returns that in-memory representation.
class ApplicationRecord < ActiveRecord::Base
self.abstract_class = true
def self.create_returning(attrs = {}, returning: "*")
record = new(**attrs)
record.attributes = insert(attrs, returning: returning).first if record.valid?
record
end
end
This approach gains back the validation check, but still leaves us without most of our callbacks (though I'd happily discourage the use of AR callbacks). I struggle to see myself using this approach because it requires abandoning standard ActiveRecord methods. That is hard enough as an active maintainer, and presents a nasty surprise for anyone new to the codebase.
Conclusion
I'd like to see Rails add an optional returning argument to the #create method. Until then, I'll probably opt for solution 2 or 1. If 2 isn't feasible for some reason and the second roundtrip required by 1 is untenable, I'd probably opt for some version of 3 with the details buried in a domain object or service object.
If you enjoyed this post, join my newsletter to get a direct line on more stuff like this.
Huge shoutout to Dillon Hafer who had a long conversation with me that led to this post and was the inspiration behind many of these solutions.
References:
Top comments (2)
Awesome post.
Excellent post, Josh! Thanks for clearly explaining this problem and your solutions.