Discussion on: Rspec or Minitest for testing Rails apps

[Vinicius Stock]

Here are some of my opinions about both frameworks.

Rspec

Pros

• Test are very easy to read and figure out what they are trying to verify
• CLI is simple. You can easily run a specific example (even if the example is defined using metaprogramming)
• The tools for stubbing and verifying method calls are simple and intuitive
• Creating a matrix of different scenarios is easy through shared contexts, lets and the example hooks (before, after, around)

Cons

• Takes longer to learn the DSL since it is not pure Ruby
• Tests take longer and have a larger memory footprint
• Requires libraries for doing things that come out of the box with minitest (such as fixtures)

Minitest

Pros

• Runs much faster than rspec
• Tests are written in pure Ruby
• Outputs not only number of examples, but also number of assertions
• Comes with fixtures and many Rails assertions out of the box

Cons

• CLI isn't the best. Hard to run a specific example
• Stubbing or verifying method calls is not as intuitive as in rspec
• Harder to create a matrix of scenarios since you cannot nest contexts as in rspec

[Thomas Ruggeri]

I totally agree about the matrix of scenarios. I'm still not satisfied with contexts in Rspec because of the nested nature. This is somewhat mitigated by shared examples, but still.

Excellent comparison list!

[jmc]

FWIW, there are shared contexts too, as you probably know. I'd sometimes make a regular loop outside of the context in order to repeat tests over some values. And helper functions are often handy, and multiline lets.

Beyond that, IME you just get to a point where they're dry enough and let it go. Since they're tests, they're not likely to need much maintenance, and what they do is often repetitive by nature.

Overall, I like the RSpec DSL... my main gripe was that there are two versions of it, so it's possible for a codebase to have a mix of styles. Probably not an issue for projects started after version 3 came out, though.

I haven't used minitest much, but I will say that I'm not a huge fan of structuring tests as classes (a la their "unit test" style, or Python's built-in unittest, or the various xUnit frameworks).

Running tests by line number is a nice feature of RSpec's but I've tended to use the substring filter more anyway, and it looks like minitest has one. (Often the line number changes as you make changes to the file.) Had I not been running my Ruby app inside a VM, I'd probably have made more use of editor integration there anyway.

Also it's hard to overstate the importance of faster test runs :p