loading...

Rails: Minitest results output in TAP-format for Heroku CI

davidwessman profile image David Wessman ・2 min read

When using Heroku CI for automatic testing, the default output is just pass or fail without any parsing of how many tests passed or what errors where found.

This can be improved if you output the results in the Test Anything Protocol-format (TAP-format) according to Heroku.

This format is not built into the test frameworks used in Ruby on Rails. Therefore I had to write my own test reporter.

I chose to implement this for Minitest and decided to use the gem
minitest-reporters.

Expected outcome

I have a small project where the test suit looks like this:
Output of test results without formatting

To adhere to the TAP-format I need it to also output:

TAP version 13
1..7
ok 1 - AccountsControllerTest - test_renders_users_account
ok 2 - AccountsControllerTest - test_require_login
ok 3 - CallbacksMessagesControllerTest - test_handle_status_callback
ok 4 - MessagesControllerTest - test_should_create_message
ok 5 - ContactsControllerTest - test_should_create_contact
ok 6 - ContactsControllerTest - test_should_update_contact
ok 7 - ContactsControllerTest - test_show_contact

Solution

I created a custom reporter called TapReporter by inspecting a standard reporter from the minitest-reporters gem called ProgressReporter (see source):

Output of test results with TAP-formatting

The solution on Heroku

A failing test

A failing test on Heroku

Successful tests

Successful tests on Heroku

Keywords

Discussion

markdown guide