DEV Community: Nate Shoemaker

Ruby on Lambda: Integrating with ActiveStorage

Nate Shoemaker — Tue, 11 Aug 2020 20:00:51 +0000

This article is part of our Ruby on AWS Lambda blog series. A recent project had us migrating an existing PDF document processing system from Rails Sidekiq to AWS Lambda. The processing includes OCR, creating preview images, splicing the PDF, and more. Moving to Lambda reduced processing time by 300 times in some cases.

This series of articles will serve less as a step-by-step process to get OCR serverless infrastructure up and running and more of a highlight reel of our "Aha!" moments. In part one, we talk about creating a AWS Lambda Layer with Docker. In part two, we chatted about architecting a serverless app. In part three, we went through some strategies surrounding deployment. Here in part four, we'll investigate what's needed to integrate a Rails app that uses Active Storage with AWS Lambda. Check out the other posts in the series:

Note: there are some assumptions made in this article regarding architecture and reading Part Two of this series, Planning & Architecting, may be useful before diving in here.

Performance was our main reason for moving from Sidekiq to AWS Lambda. We used Active Storage on a few models and wanted to continue using it after the migration. Specifically, we wanted to take advantage of Active Storage variants via a GraphQL API and React frontend. The app had multiple views that requested all of the variants at once, and because the variants are created at the time of the request, the user experience under Sidekiq was very poor. What if we could create all of those variants on Lambda as we processed the document? It sounded plausible to us, so we spiked on it.

Note that we'll be touching on specific pieces of code that directly relate to the integration; this won't be a tutorial on how to use Active Storage or setup AWS Lambda. We are using S3 as our storage service, and will be referencing it as a standalone entity. Let's dive in!

Uploading

Instead of passing form params to a model, we’ll take more of a manual approach. In the controller:

@document.file.upload(file)

We have a Document model with has_one_attached :file. This allows the file to be uploaded to S3 without creating the backing Active Storage record. Delaying that allows us to process the document on Lambda, create the needed variants, and respond back with data that will be used to create those records.

Lambda has no knowledge of the Rails app or Active Storage, but if we can point it to the file we just uploaded, it can reference and process it. After a Document has been created, we send document.file.key to the vent function (LINK TO PLANNING AND ARCHITECTURE POST) that starts the processing work. The document’s key is the name of the file on S3. On Lambda, the real fun begins.

Variant Creation

Essentially, we needed to trick Active Storage. When requesting a variant, Active Storage will look for variant.jpg in a specific location on S3:

"variants/#{key}/#{variant_key}"

key is the original file's key (document.file.key), and variant_key is a hash derived from the variant's transformation options (e.g. {auto_orient: true, resize: '1000x1000'}) and the Rails app's ENV['SECRET_KEY_BASE']. Let's look at the Lambda code:

def create_variant(image, key, transformation)
 # create message verifier and variant key
 verifier_key = verifier.generate(transformation, purpose: :variation)
 variant_key = Digest::SHA256.hexdigest(verifier_key)

 tmp = Tempfile.new('variant.jpg')

 MiniMagick::Tool::Convert.new do |convert|
   convert << image.path
   # apply transformations here, like:
   # convert.auto_orient
   # convert.resize transformation[:resize]
   convert << tmp.path
 end

 s3 = Aws::S3::Client.new
 s3.put_object(
   bucket:       "some-cool-bucket",
   key:          "variants/#{key}/#{variant_key}",
   body:         tmp,
   content_type: 'image/jpeg'
 )
end

#create_variant takes a File, document.file.key, and the transformation options as arguments. The variant_key is created (we'll look at what verifier is soon), and the actual image conversion is handled with ImageMagick. Note the actual ImageMagick conversions depend on the transformation options being passed in. For an example I used the options that were previously mentioned, {auto_orient: true, resize: '1000x1000'}. After the image is processed, we then place it on S3, using the specific path Active Storage expects.

Figuring out how to create the verifier_key was quite difficult. It took a late night fueled by coffee and spelunking into the bowels of Active Storage, but it proved fruitful. Let's take a look at #verifier:

def verifier
 key_generator = ActiveSupport::CachingKeyGenerator.new(
   ActiveSupport::KeyGenerator.new(ENV['SECRET_KEY_BASE'], iterations: 1000)
 )
 key = key_generator.generate_key('ActiveStorage')
 ActiveSupport::MessageVerifier.new(key)
end

Active Storage was created by people much smarter than I, and I'll be the first to admit that I did not look at all the implementation details surrounding ActiveSupport::CachingKeyGenerator, than I, and I'll be the first to admit that I did not look at all the implementation details surrounding ActiveSupport::KeyGenerator, ActiveSupport::MessageVerifier. However, their names are descriptive and give us a good bird's-eye view. A key is generated from your Rails app's SECRET_KEY_BASE, and that key is then turned into a digest that is used to create the path to the variant. A few notes:

The iterations option when instantiating ActiveSupport::KeyGenerator does not have to be 1,000. I, of course, didn't test every number, but the natural numbers I tested all had the same result. Active Storage uses 1,000, so I thought I would too.
The argument passed to #generate_key is the salt, and for our intended purposes on Lambda it can be any string.

Data for the `active_storage_attachments` record

Early on in the spike, we hit a lot of ActiveStorage::IntegrityErrors. While you can find yourself raising this error on accident in many scenarios, the reason why is always the same: the hashed contents of the file don't match what's stored in the database. Since we uploaded the file straight to S3, we purposefully didn't create a record in active_storage_attachments, and don't have the reference to the file anymore without downloading it (which would be a waste). We had to create the digest on Lambda while we still had a reference to the file.

Here's the data we collect for each page in a document (note that file is a File object):

{
 content_md5:  Digest::MD5.file(file).base64digest,
 byte_size:    File.size(file).to_s,
 key:          key,
 content_type: content_type
}

After every page is processed, our sink function collects all the data and sends a JSON response back to the Rails app.

All together now

When the Rails app receives the JSON, Sidekiq workers are started that process the response. Part of the workers' responsibility is to create a record in active_storage_attachments:

def create_blob(data)
 params = {
   key:          data['key'],
   filename:     data['key'],
   checksum:     data['content_md5'],
   byte_size:    data['byte_size'],
   content_type: data['content_type']
 }

 ActiveStorage::Blob.create_before_direct_upload!(params)
end

#create_before_direct_upload! allows us to create the record and not upload the file to S3 as it already exists there. Tricking Active Storage, and all.

And with that, Active Storage works just as it normally would. With our implementation, however, we process all variants in parallel, improving load times and user experience for everyone!

Ruby on AWS Lambda: Package & Ship It

Nate Shoemaker — Tue, 11 Aug 2020 20:00:47 +0000

This article is part of our Ruby on AWS Lambda series. A recent project had us migrating an existing pdf document processing system from Rails/Sidekiq to AWS Lambda. The processing includes OCR, creating preview images, splicing the pdf, and more. Moving to Lambda reduced processing time by 300% in some cases; parallelization for the win!

This series will serve less as a step-by-step process to get OCR serverless infrastructure up and running and more of a highlight reel of our "Aha!" moments. In part one, we talked about creating an AWS Lambda Layer with Docker. In part two, we chatted about architecture. Here in part three, we'll go through some strategies surrounding deployment. Check out the other posts in the series:

Deploying to Lambda gets more complicated as the application grows. There is a big difference between deploying a single hello world function with zero dependencies, and deploying multiple functions in multiple languages along with dependencies in a Lambda Layer. We discovered some useful things along the way while making that transition. Let's dive in.

Function naming conventions and environment variables

We followed a straightforward convention for differentiating between function names and their environments:

{function_name}_{environment}

Given our three Rails environments (development, staging, and production) and a function name of pdf_processor, we'd have three separate functions: pdf_processor_development, pdf_processor_staging, and pdf_processor_production. Adding more functions may seem a bit overwhelming in terms of function count, but the clear distinctions between functions and their environments allow for isolated development, testing, and debugging.

Along with other normal environment variables that the functions use, we added a RUBY_ENV environment variable that was either development, staging, or production. This is primarily used for name-spacing S3 buckets, DynamoDB tables, and calling other lambda functions. This decision played out well because we now have isolation in how we use other systems and services. This also makes reading logs and tracking performance metrics a lot easier.

Instrumentation

Instrumentation refers to the measure of a product's performance, to diagnose errors, and to write trace information. Early on in development, we added instrumentation to gain insight into how the bulk of the processing work performs.

Here’s our implementation:

# instrument.rb

module Instrument
 def instrument(tag)
   start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
   -> { puts("--> #{tag} took #{((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000).round}ms") }
 end
end

Then it is used like:

require 'instrument'

i = instrument('doing some work')
do_the_work()
i.call

The log would look like:

--> doing some work took 300ms

The start variable gets caught in a closure and is available for the Proc that is returned. The semi-cryptic looking Process.clock_gettime is used to read from the OS clock, and is more accurate than Ruby's Time class.

Serverless

When we first started building on Lambda, we deployed the AWS way: zip the code up and ship it via the AWS CLI. This was fine in the beginning, but it left a lot to be desired. It puts the responsibility on the developer to know about the deployment process, something that should be handled by configuration. That added responsibility also detracted from a deployment being deterministic. If you missed a small detail, a deployment, and thus the function itself, could be botched. We don't zip and deploy our Rails apps up manually, and we shouldn't have to do that with our serverless functions either!

In comes Serverless Framework. Everything we love about configuration and deterministic deployments are handled by Serverless. All configuration is written in YAML and is concise, intuitive, and incredibly powerful.. Environment variables, runtime specification, memory configuration, AWS roles, and layer versioning can all be done in a single YAML file. These features became even more valuable when we rewrote our vent function in JavaScript. We now have a separate serverless config for each lambda function that allows us to achieve true language isolation. If you'd like to learn more about using Serverless, take a look at their AWS quickstart guide.

Shipping a Lambda application doesn’t have to be difficult. I hope what we’ve learned has helped you!

Ruby on AWS Lambda: Planning & Architecting

Nate Shoemaker — Tue, 11 Aug 2020 17:58:39 +0000

This article is part of our Ruby on AWS Lambda blog series. A recent project had us migrating an existing pdf document processing system from Rails Sidekiq to AWS Lambda. The processing includes OCR, creating preview images, splicing the pdf, and more. Moving to Lambda reduced processing time by 300% in some cases; parallelization for the win!

This series will serve less as a step-by-step process to get OCR serverless infrastructure up and running and more of a highlight reel of our "Aha!" moments. In part one, we talked about creating an AWS Lambda Layer with Docker. In part two, we'll chat about architecting a serverless app. Check out the other posts in the series:

The Problem

Rails devs, tell me if you've been here before: you have small units of work that should be processed concurrently, so you reach for Sidekiq. Solved! Well, usually solved. Sidekiq is an amazing tool that we use at Hint, and it solves most concurrency problems.

However, a bottleneck is always a possibility. In our case, the document processor we engineered could process a PDF with one page, or 500. But since each page has to be processed and there is no page limit when uploading, it was common to process hundreds of pages. This work took much too time with Sidekiq, even with tens of workers. We wanted to see the same performance whether the PDF a user uploads has 50 pages, or 500.

The Solution

AWS Lambda was a good solution for our performance problem. It allows us to run our workers in parallel independent of the number of pages we are processing. However, Lambda doesn't give you any guidance on how to design a serverless application. You are provided the basic building blocks: a function that calls other functions, and the ability to use any other AWS service. Quite open-ended! Luckily, there are some standard architecture practices that have emerged in the serverless community. We'll be focusing on the most popular strategy: fan-out, fan-in.

"What do you mean, software architecture?"

Glad you asked! Martin Fowler said it best:

When people in the software industry talk about “architecture”, they refer to a hazily defined notion of the most important aspects of the internal design of a software system.

When we talk about architecture in the context of Lambda, we'll be touching on function composition, utilizing other AWS services, and how the application communicates with the real world. If you haven't, check out Martin Fowler's thoughts on software architecture.

Fan-out, Fan-in

The fan-out, fan-in architecture is simple: one function starts n number of functions, and when those finish, another function does something with the results. The first function is referred to as the ventilator (or vent). The vent function calls worker functions. Once the worker functions finish, the sink function is called. So, vent -> workers -> sink.

For example, let's use a hypothetical 100-page pdf.. The vent function is triggered from an outside application and 100 worker functions are called. Each worker process a single page. The sink function is then invoked when all 100 are done, and we do things with the result. In theory, 1,000,000 pages should take as long as 100, since all the processing is happening in parallel. Let's go into a little bit more detail about the whole process:

User uploads PDF from Rails app
PDF is uploaded to S3 via Active Storage
Vent function is triggered with the file key as an argument
Vent function calls n number of worker functions, passing file key as an argument so workers can find the file on S3
Worker function processes page, stores preview image, collects information about page and stores that in DynamoDB, and a record in DynamoDB that holds the keeps track of the pages left is decreased by one
If the worker is the final worker in the queue, the sink function is called
Sink function collects all page information from DynamoDB and sends results as JSON back to the Rails app

DynamoDB is a NoSQL database offered by AWS. When using the fan-out, fan-in pattern you must keep track of progress so the sink function can be called when all workers are completed. The nature of serverless means that no state can be held, thus an external, stateful service must be used. In our case, we are using it as a glorified counter. When a worker function finishes, we decrease the records count column (pages_left) by one:

  def decrement_pages_left(job_id)
    db = Aws::DynamoDB::Client.new
    resp = db.update_item(
      table_name:                  "documents-#{ENV['RUBY_ENV']}",
      key:                         { job_id: job_id },
      update_expression:           'set pages_left = pages_left - :val',
      expression_attribute_values: { ':val' => 1 },
      return_values:               'UPDATED_NEW'
    )

    invoke_sink_function(job_id) if resp[:attributes]['pages_left'] == 0
  end

Note that the sink function is then called if there are no pages left to process. Easy!

For a more in-depth look into the fan-out, fan-in pattern take a look at this fantastic blog post by Yan Cui.

Knowing what language is better suited for a certain task

We initially used Ruby for our vent, worker, and sink functions. However, we hit a bottleneck in the vent function. Invoking hundreds of concurrent network requests (which are IO bound) is not one of Ruby's strong suits. Using any kind of library that takes advantage of CPU cores doesn't gain any advantage because of Lambda's CPU limitations (more on that in the next section). So, what language does Lambda support that has asynchronous IO bound operations out of the box? JavaScript!

Now, this isn't a bash on Ruby. All programming languages have their strengths and weaknesses, and a rewrite in a different language should be well researched and thought out beforehand. Luckily for us, we have lots of Ruby and JS experience. Also, the vent function encapsulates very little business logic, so a rewrite would be a good fit if an initial spike proved fruitful. And it did! When processing a 186-page document, the vent function took 30 seconds in Ruby, and 2 seconds in JS. Nailed it!

If you experience performance issues on Lambda, make sure to research the problem thoroughly. The cool thing about Lambda is that you can use different languages throughout the application process. If part of the process would perform much better with a different language, try it out!

CPU/RAM limitations

When assigning resources to your Lambda functions, memory is the only configurable option. Why? Well, there is no obvious answer in the UI, and even worse, it's hidden deep in a FAQ. When you choose the amount of memory you want to allocate for a function, you are given proportional CPU power and other resources. What the docs don't tell you, however, is that if you allocate enough memory, you'll be given two cores instead of one.

This Stack Overflow comment has more info. TL;DR: if you are using 1.8GB or more of memory, you get to use two CPU cores, instead of one. Even two cores is not a lot, and that's why asynchronous operations in programming languages that utilize multiple cores don't perform well on Lambda. When architecting a serverless application, it's better to split larger tasks into smaller subtasks, when possible.

I hope you learned something new today! Architecting serverless applications come with their own unique set of challenges, but the community has great solutions that have been production-tested for quite some time.

Macro See, Macro Do

Nate Shoemaker — Fri, 17 Apr 2020 14:38:51 +0000

Originally posted on Hint's blog.

This post assumes you have Erlang/Elixir installed and that you can spin up a Phoenix project.

I've been diving into Elixir and Phoenix as of late. I have thoroughly enjoyed my time spent with both and could gush about them for far, far too long. Elixir has some features whose underlying implementation wasn't obvious at first glance, namely the macro quote/2 (that slash and number indicates the arity).

I first came across quote/2 in a fresh Phoenix project. Let's create one now! Run mix phx.new foo, open up lib/foo_web/foo_web.ex and see quote/2 being used:

# lib/foo_web/foo_web.ex

defmodule FooWeb do
  def controller do
    quote do
      use Phoenix.Controller, namespace: FooWeb

      import Plug.Conn
      import FooWeb.Gettext
      alias FooWeb.Router.Helpers, as: Routes
    end
  end

  # ...
end

controller/0 is then used in controllers like this:

# lib/foo_web/controllers/page_controller.ex

defmodule FooWeb.PageController do
  use FooWeb, :controller

  # ...
end

Going through Programming Phoenix I saw this macro being used again and again. I understood what it meant: the use, import, and alias macros are being injected into PageController, so dependencies can be shared across modules. But why not just include them in the function definition? What is going behind the scenes? Why quote/2? Being a Rails developer accustomed to magic, I accepted it and moved on.

One of Phoenix's (and Elixir) strengths is that nothing is hidden from the developer. Everything is gloriously defined, displayed, and explicitly composed right in front of you. There really isn't any magic. Thus my acceptance of it bothered me, so let's dive in and learn about quote together!

Copy and (almost) paste

The best way to learn is by doing, so why don't we create some modules and reproduce what we've seen. Here's a very simple example I came up with:

# bar.exs

defmodule Bar.Math do
  def sum(x, y), do: x + y
end

defmodule Bar.AllTheThings do
  def things do
    quote do
      alias Bar.Math
    end
  end
end

defmodule Bar.Work do
  use Bar.AllTheThings, :things

  def print_sum(x, y) do
    IO.puts("the sum of #{x} and #{y} is #{sum(x, y)}")
  end
end

Bar.Work.print_sum(2, 2)

Again, I don't really know what quote/2 is doing and why, but we mimicked what we saw in Phoenix pretty close. I think we're ready to try this out, let's run elixir bar.exs and see what happens:

> elixir bar.exs
** (UndefinedFunctionError) function Bar.AllTheThings.__using__/1 is undefined or private
    Bar.AllTheThings.__using__(:things)
    bar.exs:18: (module)
    bar.exs:17: (file)

🤔

It seems that we're missing a function that Elixir assumes we have implemented. I'll be honest - I've never written a module that was consumed by use, so let's double back to FooWeb in our Phoenix App to see if we missed anything. At the bottom of the file, you'll see:

  defmacro __using__(which) when is_atom(which) do
    apply(__MODULE__, which, [])
  end

Ah! Elixir was looking for that function, so let's slap that it in Bar.AllTheThings:

defmodule Bar.AllTheThings do
  def things do
    quote do
      alias Bar.Math
    end
  end

  defmacro __using__(which) when is_atom(which) do
    apply(__MODULE__, which, [])
  end
end

Diving into defmacro is outside the scope of this post, but we can acknowledge it as a requirement of a module that's consumed by use. The use of apply/3 is straightforward: take a module, an atom that represents the function name, and call it with some arguments.

apply(Bar.AllTheThings, :things, [])
# is equivalent to
Bar.AllTheThings.things([])

And then:

> elixir bar.exs
the sum of 2 and 2 is 4

Great, our dependency injection works. Now that we understand the structure let's dig into what's happening under the hood.

I heard you like Elixir, so let's represent some Elixir with Elixir

From the docs:

> quote(opts, block)

Gets the representation of any expression.

Let's try it out:

> iex
iex(1)> quote do sum(2, 2) end
{:sum, [], [2, 2]}

That's right! We are representing Elixir with Elixir. Elixir's AST (abstract syntax tree) is... Elixir! Pretty cool, huh? Macros, such as quote/2, are represented by a tuple of three elements. The first element is (usually) an atom, the second is for metadata, and the third is the argument list.

I wonder what our import Bar.Math looks like as an AST? Let's find out! Comment out everything in bar.exs except for the Bar.Math module. Rename the file to bar.ex so Elixir can compile it, and run iex:

> iex
iex(1)> c "bar.ex"
[Bar.Math]
iex(2)> quote do
...(2)>   import Bar.Math
...(2)> end
{:import, [context: Elixir], [{:__aliases__, [alias: false], [:Bar, :Math]}]}

There it is! We can see the AST as a three element tuple. It holds all the information that Elixir needs to know to import a module. quote/2 gives us some fantastic syntax sugar; could you imagine writing these tuples everywhere? Just for fun, let's see how deep into the rabbit hole we can go. Rename bar.ex back to bar.exs, uncomment all the code, and change the import Bar.Math to the AST representation without quote/2:

# bar.exs

# ...
defmodule Bar.AllTheThings do
  def things do
    {:import, [context: Elixir], [{:__aliases__, [alias: false], [:Bar, :Math]}]}
  end

  defmacro __using__(which) when is_atom(which) do
    apply(__MODULE__, which, [])
  end
end
# ...

And:

> elixir bar.exs
the sum of 2 and 2 is 4

It works! Let's go another level in by removing things/0 and placing our AST directly in __using/1:

# bar.exs

# ...
defmodule Bar.AllTheThings do
  defmacro __using__(which) when is_atom(which) do
    {:import, [context: Elixir], [{:__aliases__, [alias: false], [:Bar, :Math]}]}
  end
end
# ...

You know the drill:

> elixir bar.exs
the sum of 2 and 2 is 4

Nice! Is it possible to inline the AST we have in our Bar.Work module? Sadly, we can't. The use/2 macro changes this:

use Bar.AllTheThings, :things

to:

Bar.AllTheThings.__using__(:things)

We've come to the end of this Elixir in Elixir train! There are no other stops on this line.

Wrapping up

So, what did we learn? The quote macro transforms Elixir code to an AST. We can then leverage that to achieve real dependency injection in a functional language. How cool is that?

Easily Integrate Ramda into Your React Workflow

Nate Shoemaker — Fri, 13 Sep 2019 19:45:34 +0000

Originally posted on Hint's blog.

Here at Hint, we often use React for writing our user interfaces. We enjoy its declarative API, the mental-model that makes it easier to communicate and collaborate with teams, and especially, the recent addition of hooks. React doesn't provide the entire toolkit, however. It's missing a few things out of the box: data fetching, handling async functions, applying styles in a pragmatic way, etc.

As I was learning React, the biggest hole in React's feature set actually turned out to be an issue with JavaScript itself. Compared to other toolkit heavy languages such as Ruby or Elixir, JavaScript doesn't give you a ton to work with. I started writing my own helper libraries until a friend told me about Ramda. Straight from their homepage:

A practical functional library for JavaScript programmers.

Hey! I like functional things, libraries, JavaScript... and I'm a programmer! It was love at first byte (no, I don't feel any shame for that).

The first Ramda hurdle is functional programming. If you have never dipped a toe in the functional waters, please read Randy Coulman's "Thinking in Ramda" series, it's brilliant.

The second Ramda hurdle (as a React developer) is knowing how to use it with React effectively. I'm still learning and experimenting with how the two libraries can work together, and I wanted to share some of the patterns that I have held onto over the past few years. Let's get into it!

Make Your Code Read Better With `isNil` And `isEmpty`

Sometimes, React code isn't the easiest to read. I would argue that post-hooks this has gotten even worse. More and more logic is being added to the component's body, and without lifecycle methods that automatically help organize code out of render, any help I can get to cleanup, I take.

Ramda's isNil and isEmpty are a great start to make your component's body dazzle 🕺. For example:

  const Entry = ({ client }) => (
    <Query query={currentUserQuery}>
      {({ loading, data }) => {
        if (!loading && !data.user.posts)
          return <NoPosts />

        if (data.user) {
          setErrorTrackingContext(data.user)
          getPostMetaData(data.user, client)
        }

        return (
          // code that renders things here
        )
      }}
    </Query>
  )

Note on code examples: all code in this article is based on real life code that I've written. There are some references to Apollo's React library, which Hint loves. Most imports have been removed for brevity. No blogpost-ed, fooBar-filled, faux-code here. Nearly Production Ready™.

Note the first if: we'll return a component early if we're done loading and the data.user.posts is falsy. The second if: if we have a user, let's set the context for whatever error tracking we're using (at Hint we love Honeybadger), then get some post metadata. Let's not worry about any implementations of these functions and focus on our logic. At first glance, things aren't that bad - but "not that bad" is not the bar. Excellence is! Let's take another pass, but with Ramda:

  import { isNil, isEmpty } from 'ramda'

  const Entry = ({ client }) => (
    <Query query={currentUserQuery}>
      {({ loading, data }) => {
        if (isNil(loading) && isEmpty(data.user.posts))
          return <NoPosts />

        if (data.user) {
          setErrorTrackingContext(data.user)
          getPostMetaData(data.user, client)
        }

        return (
          // code that renders things here
        )
      }}
    </Query>
  )

Note the import at the top and the update to our first if. isNil will return true if loading is null or undefined. This function is extremely helpful because it doesn't just check if the value is falsy, which is essentially what it did before (!loading). Hindquarters saved from a nasty bug!

On the same line, isEmpty will return true if the value passed in is '', [], or {}. When working with GraphQL, if you ask for a collection of things but there are none, more often than not you'll get back an empty array. Our logic check before, !data.user.posts could have also introduced an unintended bug! Hindquarters saved AGAIN.

Pro-Tip

First point and already a pro-tip? Today is a good day.

Ramda is built of many tiny functions that have a single specific purpose. Assembled together properly, you can create some fun stuff! Let's create a helper that's the inverse of isNil:

  import { isNil, isEmpty, complement } from 'ramda'

  const isPresent = complement(isNil)

  const Entry = ({ client }) => (
    <Query query={currentUserQuery}>
      {({ loading, data }) => {
        if (isNil(loading) && isEmpty(data.user.posts))
          return <NoPosts />

        if (isPresent(data.user)) {
          setErrorTrackingContext(data.user)
          getPostMetaData(data.user, client)
        }

        return (
          // code that renders things here
        )
      }}
    </Query>
  )

complement takes a function as its first argument, and a value as its second. If a falsy value is returned when it's called, the output will be true (the inverse is also true). Using complement makes our second if a little nicer.

You may say, "Well that's really simple. Why doesn't Ramda come with a helper like that?" Think of Ramda functions like individual LEGOS pieces. On their own, they don't do a ton, but put them together, and you can create something incredibly useful. If you want a more "comprehensive set of utilities", check out Ramda Adjunct.

It's Dangerous to Operate on Objects Alone! Take These Functions: `prop` and `path`

+1 internet points if you get the title joke

As a developer, nothing is more scary than deeply accessing an object. If this doesn't make you slightly cringe:

if (foo.bar.baz.theLastPropertyIPromise.justKiddingOneMore) doTheThing()

Then we need to have a talk. If this is your proposed solution:

if (
  foo &&
  foo.bar &&
  foo.bar.baz &&
  foo.bar.baz.theLastPropertyIPromise &&
  foo.bar.baz.theLastPropertyIPromise.justKiddingOneMore
)
  doTheThing()

Then we really need to talk.

Joking aside, we've all been there. It's easy to gloss over complex checks completely or write conditionals that take up too many bytes and are difficult to read. Ramda gives us prop and path to safely access objects. Let's see how they work:

import { prop, path, pipe } from 'ramda'

const obj = { foo: 'bar', baz: { a: 1, b: 2 } }

const getFoo = prop('foo')
getFoo(obj) // => 'bar'

const getBazA = path(['baz', 'a'])
getBazA(obj) // => 1

Great! "But what about that is safe? All the properties you asked for are present!" Glad you asked:

import { path, pipe } from 'ramda'

const obj = { foo: 'bar', baz: { a: 1, b: 2 } }

const getSomethingThatDoesNotExist = path([
  'foo',
  'bar',
  'baz',
  'theLastPropertyIPromise',
  'justKiddingOneMore'
])
getSomethingThatDoesNotExist(obj) // => undefined

Thanks Ramda! Hindquarters, yet again, saved. Note that undefined, a falsy value is returned. Very useful for presence checks! Let's apply our new learnings to our <Entry /> component:

  import { isNil, isEmpty, complement, prop } from 'ramda'

  const getUser = prop('user')
  const userIsPresent = pipe(
    getUser,
    complement(isNil)
  )

  const Entry = ({ client }) => (
    <Query query={currentUserQuery}>
      {({ loading, data }) => {
        if (isNil(loading) && isEmpty(data.user.posts))
          return <NoPosts />

        if (userIsPresent(data)) {
          const user = getUser(data)
          setErrorTrackingContext(user)
          getPostMetaData(user, client)
        }

        return (
          // code that renders things here
        )
      }}
    </Query>
  )

Looking better for sure. Further refactoring could be done in our second if condition. For fun, see if you can figure out how to use Ramda to bring that if into one function. Answer is at the end of this post!

Prep Your Props With `evolve`

Transforming component props into something useful is common practice. Let's take a look at this example where we concat a first and last name as well as format a date:

const NameAndDateDisplay = ({ date, firstName, lastName }) => (
  <>
    <div>
      Hello {firstName.toUpperCase()} {lastName.toUpperCase()}!
    </div>
    <div>It is {dayjs(date).format('M/D/YYYY dddd')}</div>
  </>
)

Straightforward, but there is something fishy about this code, though. Can you spot it? The problem is that it's a little too straightforward. When working with real data, real API's, and real code that humans have written, things aren't always straightforward. Sometimes you are working on a project that consumes a third-party API and you don't have full control on what you get back from the server.

In these cases, we tend to throw all of our logic in our component bodies, like so:

const NameAndDateDisplay = ({ date, firstName, lastName }) => {
  const formattedDate = formatDate(date)
  const formattedFirstName = formatFirstName(firstName)
  const formattedLastName = formatLastName(lastName)

  return (
    <>
      <div>
        Hello {firstName} {lastName}!
      </div>
      <div>It is {formattedDate}</div>
    </>
  )
}

This presents a few issues. Some very important logic is tied to the body of our component, making testing difficult. The only way to test those formatters is to render the component. Also, it's really bloating the body of our component. In Rails you'll here "Fat models, skinny controllers"; an analogous term in React would be "Fat helpers, skinny component body".

Luckily, Ramda's evolve can really help us out. evolve takes two arguments; the first is an object whose values are functions, and the second argument is the object you want to operate on.

import { evolve, toUpper } from 'ramda'

evolve({ foo: toUpper }, { foo: 'weeee' })
// => { foo: 'WEEEE' }

Pretty neat! Two important things to note about evolve: it's recursive and it doesn't operate on values you don't specify in the first argument.

import { evolve, toUpper, add } from 'ramda'

const format = evolve({
  foo: toUpper,
  numbers: { a: add(2) },
  dontTouchMe: 'foobar'
})
format({ foo: 'weeee', numbers: { a: 3 } })
// => { foo: 'WEEEE', numbers: { a: 5 }, dontTouchMe: 'foobar' }

With this newfound knowledge, let's refactor our component:

import { evolve, pipe } from 'ramda'

const prepProps = evolve({
  date: formatDate,
  firstName: formatFirstName,
  lastName: formatLastName
})

const NameAndDateDisplay = ({ date, firstName, lastName }) => (
  <>
    <div>
      Hello {firstName} {lastName}!
    </div>
    <div>It is {date}</div>
  </>
)

export default pipe(
  prepProps,
  NameAndDateDisplay
)

Sick! We have successfully split our formatting code away from our rendering code.

Wrapping Up

React and Ramda are both incredibly powerful tools. Learning how they work and interact together can simplify and speed up development time.

Going forward, keep Ramda in mind when you find yourself copying & pasting helper libraries from one project to the next. Odds are, a Ramda function exists that can accomplish the same task, and more! There are many, many more Ramda functions not covered in this article. Look to Ramda's documentation to learn more.

Refactoring Answer

Our second if condition, fully refactored:

// setErrorTrackingContextAndGetPostMetaData.js
import { prop, pipe, complement, when, converge, curry, __ } from 'ramda'

const getUser = prop('user')
const userIsPresent = pipe(
  getUser,
  complement(isNil)
)
const curriedGetPostMetaData = curry(getPostMetaData)

const setErrorTrackingContextAndGetPostMetaData = client =>
  when(
    userIsPresent,
    converge(getUser, [
      setErrorTrackingContext,
      curriedGetPostMetaData(__, client)
    ])
  )

export default setErrorTrackingContextAndGetPostMetaData

// Entry.js
// in the body of <Entry />

// ...
setErrorTrackingContextAndGetPostMetaData(client)(data)
// ...