DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’» is a community of 966,904 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

Create account Log in
Stanislav(Stas) Katkov
Stanislav(Stas) Katkov

Posted on

Fully automated automated testing (doom guy edition)

Woohoo, Doom guy in notifications!

I hate repetitive work with passion and I try to avoid it as much as possible. While browsing twitter I randomly stumbled on Sam Saffron video, there he talks about his love of improving own workflow and building dev tools.

One off first point Sam mentions there is autospec and demonstrates how it works. I had really mixed feelings about this particular part. I felt relieved that my testsuite takes less than 5 minutes, but at the same time, I had this nagging feeling inside me too...

My workflow consists of following stages:

  1. do small code changes
  2. run tests
  3. rollback or fix test if something failed
  4. continue to step 1

It's hard to believe, but most developers do this all day. Some people manually execute tests, but I just run automated tests manually. In my case it takes 20 second for ~500 assertions to happen... It feels repetitive really fast.

So... And here is Sam... He doesn't execute testsuite routinely through the day? Doesn't pick specific files for testing? Yes, please, give me the same. But I felt even worse then I understood that autospec doesn’t work with minitest. I had no desire to change to rspec.

So what do other minitest users do in this case? What would you do? If your using guard already, you may want to pick guard-minitest. But it just runs all tests, without focusing on broken one. If it is okey with you - then go on and use it. if you want something better you can read further.

Let's use minitest-autotest

It’s pretty easy to add. Just open your Gemfile and add these lines:

    group :test do
      gem "minitest-autotest”
    end
Enter fullscreen mode Exit fullscreen mode

Create a simplest possible configuration file in root of your project and call it .autotest

    require 'autotest/restart'

    Autotest.add_hook :initialize do |at|
      at.testlib = "minitest/autorun"
    end
Enter fullscreen mode Exit fullscreen mode

You don’t really need it, it β€œjust works” (c) without it. But we'll need it later, there is obviously something missing here. I don’t want to be checking my console all the time, to be sure that I didn’t broke anything.

Better notification to the rescue!

We already talked about guard. Guard wiki contains very nice summary about system notifications and libraries that we can use. Since I use OSX - my two options are growl and terminal-notifier. I don't really feel like paying for growl, so terminal notifier seems like a perfect candidate for a job. Let’s add more code, shall we?

Our Gemfile should have one additional line:

    group :test do
      gem "minitest-autotest"
      gem "terminal-notifier"
    end
Enter fullscreen mode Exit fullscreen mode

And our .autotest code becomes a bit more interesting:

    require 'autotest/restart'
    require 'terminal-notifier'

    Autotest.add_hook :initialize do |at|
      at.testlib = "minitest/autorun"
    end

    Autotest.add_hook :ran_command do |at|
      if at.failures.count > 0
        TerminalNotifier.notify(failed_message(at.failures.count),
          :title => 'minitest-autotest'
         )
      end
    end

    def failed_message(count)
      count.eql?(1) ?  "#{count} test failed" :  "#{count} tests failed"
    end
Enter fullscreen mode Exit fullscreen mode

This is already much better - now we receive notification every time tests fail. But this sounds like a problem:

  • we're bombarded by notifications. autotest focuses on erroneous test and runs it indefinitely (generating notifications every single time)
  • If we rollback our code, we have to again verify in console that spec's passed. Would be nice, if we don't have to do this.

Let's make notifications more intelligent (without machine learning...)

    require 'autotest/restart'
    require 'terminal-notifier'

    SKIP_MESSAGES = 5

    Autotest.add_hook :initialize do |at|
      at.testlib = "minitest/autorun"
      @counter = 0
    end

    Autotest.add_hook :all_good do |at|
      if @counter > 0
        TerminalNotifier.notify("All fixed!",
          :title => 'minitest-autotest'
        )

        @counter = 0
      end
    end

    Autotest.add_hook :ran_command do |at|
      if at.failures.count > 0
        if @counter.eql?(0) || @counter.eql?(SKIP_MESSAGES+1)
          TerminalNotifier.notify(failed_message(at.failures.count),
            :title => 'minitest-autotest'
          )

          @counter = 1
        else
          @counter += 1
        end
      end
    end

    def failed_message(count)
      count.eql?(1) ? "#{count} test failed" : "#{count} tests failed"
    end
Enter fullscreen mode Exit fullscreen mode

So now we added a simple counter. Based on this counter we can determine:

  • If we already shown error notifications and present 'Wohoo, everything is fixed' notification
  • We can skip some of notifications, not to be overly annoying

Not a cool version of notifications

It seems like this is almost perfect for my day-to-day activities. But lacks coolness..

Doom guy enters the stage..

If your my age, you will never forget about epic game called Doom. This could also be a purely Estonian problem, because Jarmo Pertman (who is my age and Estonian as I am) long time ago wrote a doom-guy-bleeding indicator plugin for old version of autotest that doesn't work nowadays.

Let's use his work for good - steal some of images he had there and place them into /vendor/doomguy/ folder. And steal some of his code (thank god it's open source). Let's sum all of this:

    require 'autotest/restart'
    require 'terminal-notifier'

    SKIP_MESSAGES = 5
    IMAGE_PATH = File.dirname(__FILE__) + "/vendor/doomguy/"

    Autotest.add_hook :initialize do |at|
      at.testlib = "minitest/autorun"
      @counter = 0
    end

    Autotest.add_hook :all_good do |at|
      if @counter > 0
        TerminalNotifier.notify("All fixed!",
          :title => 'minitest-autotest',
          :appIcon => IMAGE_PATH + "pass.png"
        )

        @counter = 0
      end
    end

    Autotest.add_hook :ran_command do |at|
      if at.failures.count > 0
        if @counter.eql?(0) || @counter.eql?(SKIP_MESSAGES+1)
          count = [(9 + at.failures.count) / 10 * 10, 50].min

          TerminalNotifier.notify(failed_message(at.failures.count),
            :title => 'minitest-autotest',
            :appIcon => IMAGE_PATH + "fail#{count}.png"
          )

          @counter = 1
        else
          @counter += 1
        end
      end
    end

    def failed_message(count)
      count.eql?(1) ? "#{count} test failed" : "#{count} tests failed"
    end
Enter fullscreen mode Exit fullscreen mode

So, now I'm fully happy with result and productive!

Woohoo, Doom guy in notifications!

Happy testing!

Top comments (4)

Collapse
 
maestromac profile image
Mac Siri

I like our approach! Do you always have test running in the background whenever you are working on a project?

Collapse
 
skatkov profile image
Stanislav(Stas) Katkov Author

That's a pretty new approach for me, just following it for a week. But yeah, tests are running on their own all the time - so far works great ;-)

Collapse
 
aliasgar1982 profile image
Aliasgar Baldiwala

Hi there! We just migrated to Git and wanted to get your take on pipeline testing tools for load and web API. Also what do you suggest for auto automating functional tests?

Thread Thread
 
dipudarl profile image
Deepali

He Ali....weird way to connect...i assume u ali from tolani bms...was trying to connect to old frds...u can contact or message me on +91 9820858841...was trying to google you and all i see a post from u here...Nishant Vora - Tolani BMS

Take a look at this:

Settings

Go to your customization settings to nudge your home feed to show content more relevant to your developer experience level. πŸ›