Unit tests in Trailblazer: less code, more coverage.

by Nick Sutterer @apotonick

Writing and maintaining automated tests for code is the most hated part for every developer. You might find yourself nodding to this as you're reading.

It's not only the pain to set up the environment needed for testing a specific case, but also the amount of actually written lines of code to make sure you're covering "everything" that could be affected by the new chunk of code you introduced.

In this post, I want to focus on the second part of this bold statement and quickly show you how the just released trailblazer-test gem allows writing very concise unit tests for Trailblazer operations. A lot of work has been put into it to make covering edge cases as simple as possible.

What's that "operation"?

Trailblazer provides a service object called operation. It's the place where you put the business logic for a particular use case you are working on. With it's very simple DSL, logic can be organized in chunks and executed step-wise.

module Memo::Operation
  class Create < Trailblazer::Operation
    step :check_data
    step :validate
    step :save
    # ...

    def save(ctx, params:, **)
      ctx[:model] = Memo.create(params[:memo])
    end
  end
end

In short, when invoking the Memo::Operation::Create operation, the three exemplary steps will be executed in the order you defined them. If a step fails (by returning false), the remaining steps are skipped and the operation terminates, indicating a failure.

If you're keen to learn more about operations and the internal railway model, check our extensive docs or, much better, simply watch a 5 minute video.

It's good practice to have full coverage of your operations as this is the place (in a Trailblazer-driven app) where the business code lives.

Minitest or RSpec?

In the examples used to illustrate the new gem, we're using Minitest, as we think it's much more readable than RSpec. RSpec on the other hand is providing a great toolset but, whatsoever, is putting too much effort into a test DSL that's extremely verbose.

As a matter of fact, this is a matter of taste - no holy wars here: we also have RSpec support.

Asserting success

Now, to show you a very simple test case, let's create a test file, configure it, and run the Create operation with a particular input to assert that it terminates successfully.

# test/operation/memo_test.rb

require "test_helper"

class MemoOperationTest < Minitest::Spec
  Trailblazer::Test.module!(self) # install our helpers.

  it "passes with valid input" do
    input = {params: {memo: {content: "Stock up beer"}}}

    assert_pass Memo::Operation::Create, input
  end
end

Using Test.module! you include the #assert_pass assertion into the test class.

#assert_pass, in its purest form, takes the operation constant, and an input hash. Internally, the assertion now runs the operation with the specified input and then tests if the outcome was successful. This roughly translates to the following snippet, something I've seen throughout many TRB projects.

it "passes with valid input" do
  # ...
  result = Memo::Operation::Create.(input)

  assert_equal result.success?, true
end

There is nothing wrong with doing the above manually, but our assertions bring a (hopefully!) much better developer experience that we're about to discover.

Debugging? You're welcome!

A typical issue for developers when writing or changing tests is that an operation supposed to be passing actually fails. Most of the times, this is due to validation errors. Given that you're using a contract with an errors object, a failing #assert_pass will automatically print out the validation errors.

Within a second, you know that your input passed to the operation is not satisfying the validations. If that is not enough, you can simply add a question mark to the assertion.

it "passes with valid input" do
  # ...
  assert_pass? Memo::Operation::Create, input
end

This gives you the highly popular #wtf? trace on the console

Those two incredibly helpful features for debugging have been suggested by several TRB users over the years, as both checking the contract errors as well as turning on tracing (#wtf?) are the first things immediately done manually by many developers when hitting an issue.

Testing the model

While checking if an operation ran successfully is a great thing to do, bringing joy and happiness to the team, the product managers, and the clients, a good test needs to do a bit more.

In most cases, an operation produces or alters a model, which is usually found under ctx[:model]. After running, you may want to check if model attributes match your high expectations.

One way would be to use the block style and do the testing yourself.

it "passes with valid input" do
  # ...
  assert_pass Memo::Operation::Create, input do |result|
    assert_equal result[:model].content, "Stock up beer"
    # ...
end

The block simply yields the result objectand it's up to you what's done inside.

Alternatively, you can use the built-in attributes test of #assert_pass.

it "passes with valid input" do
  # ...
  assert_pass Memo::Operation::Create, input,
    content:    "Stock up beer",
    persisted?: true,
    id:         ->(asserted:, **) { asserted.id > 0 }
end

Your new best friend #assert_pass takes keywords as its third argument. Those are automatically matched against result[:model]. As you can see for :id, even dynamic assertions are possible.

The combination of the block style and the built-in model assertions provides a rich interface for testing any successful outcome of your operations.

When things go wrong

Testing the scenarios where an operation passes is a wonderful thing to do. However, probably even more important is testing when things don't play and operations are supposed to actually fail. To test a failing operation, we got #assert_fail - you already guessed that method name, right?

In many cases, an operation will fail if its validations aren't met. You can simply check if the operation terminated on the failure terminus by using the new assertion with one argument, only.

it "fails with invalid input" do
  invalid_input = {params: {memo: {}}}

  assert_fail Memo::Operation::Create, invalid_input
end

In rare cases, this might be a sufficient test, but most of the times you want to assert errors more detailed.

Testing error messages

Given that you're using a [contract in the operation], you can ask #assert_fail to check for specific validation error messages.

it "fails with invalid input" do
  # ...
  assert_fail Memo::Operation::Create, invalid_input,
    [:title, :content] # erroring fields.
end

The assertion will now check if the internal contract errors object contains the erroring fields you provided, resulting in a manual test that could look like so.

it "fails with invalid input" do
  # ...
  assert_equal result["contract.default"].errors.messages.keys,
    [:title, :content]
end

To write an even stricter test, you can provide the error messages as an additional constraint.

it "fails with invalid input" do
  # ...
  assert_fail Memo::Operation::Create, invalid_input,
    {
      title: ["must be filled"],
      content: ["must be filled", "size cannot be less than 8"]
    }
end

When expected error messages do not match the actual ones, the assertion automatically shows you the latter.

Again, optimizing your experience and shortcutting ways to help you debug.

What about extendability?

Both assertions shipped with trailblazer-test provide the block syntax and return the result, in case you need to add more test code.

Also, keep in mind that the assertions described here are what we needed to minimize time, code and brain when writing tests. Feel free to ping us for discussing any extensions of the gem.

Suite: Minimizing code

The assertions described so far are designed to take away pain in your testing, but they require you to repeat arguments over and over again. The "Suite" mode targets defaulting, so the written code is even less.

Imagine you're testing our Create operation and you want to make sure that all validations are actually working, each one in a separate test case. Here's how that could look using the suite feature.

# test/memo/operation_test.rb
class MemoOperationTest < Minitest::Spec
  Trailblazer::Test.module!(self, suite: true)

  describe "Create" do
    # insert defaulting here, see below...

    it "{content} works" do
      assert_pass({content: "chill beer"}, {content: "chill beer"})
    end

    it "{tag_list} is converted to array" do
      assert_pass(
        {tag_list: "fridge,todo"},      # input
        {tag_list: ["fridge", "todo"]}  # model value.
      )
    end
  end
end

In suite mode, assertion arguments such as operation, the incoming ctx and expected attributes on the model can be set (and overwritten!) on the class and describe level.

Defaulting over verbosity

You can default arguments by simply defining special-named let() blocks on any level.

# test/memo/operation_test.rb
class MemoOperationTest < Minitest::Spec
  # ...
  describe "Create" do
    let(:operation) { Memo::Operation::Create }
    let(:default_ctx) do
      {
        params: {
          memo: { # Note the {:memo} key here!
            title:   "Todo",
            content: "Stock up beer",
          }
        }
      }
    end
    let(:expected_attributes) { ... }
  end

Instead of having to repeat those values, the suite-enabled assertions will use and accordingly merge arguments for you. A desired side-effect is that #assert_pass always checks all attributes on the model as it merges expected_attributes with the second hash you provided.

Check the docs to dive into this simple yet helpful feature.

Stop mocking me!

While it's usually good practice to test the entire stack of logic, meaning your tests also cover complex system parts like external services, sometimes it's necessary to stub a component.

Replacing a particular step can easily be done using #mock_step. You are correct when objecting that this method should be named #stub_step, but that's too close to dub_step and we haven't added an alias, yet. Note that a "step" could be an entire, nested operation using Subprocess(), anything modeled as a step can be stubbed.

it "runs fine" do
  stubbed_create = mock_step(Memo::Operation::Create, path: [:save]) do |ctx, **|
    # new logic for {save}.
    ctx[:saved] = true
  end

  assert_pass stubbed_create, ...
end

The :path option allows targeting either a first-level step sitting directly in Create, or a deeply nested step somewhere 6 levels down in your nested operation graph. Check the docs for some more detailed examples.

The #mock_step helper returns a new operation class which can then be passed to the assertions, or even returned from let(:operation) if using suite mode.

Maybe this post is a good place to mention that the entire stubbing logic is simply using the patch feature of Trailblazer internally - implementing this for the trailblazer-test gem was nothing more but applying the patching mechanics with three lines of code.

RSpec and more

In the next post we're going to introduce the RSpec matchers that are based on this gem.

it "passes with manual attributes" do
  input = {params: {memo: {title: "Reminder", content: "Do not forget"}}}

  expect(run(Memo::Operation::Create, input)).
    to pass_with(title: "Reminder")
end

Being a bit more verbose, they provide the exact same behavior that we ship for Minitest. If you have suggestions or ideas, never hesitate to discuss those with us! Now, have fun testing. Or at least, try to!