DEV Community

Cover image for Tests and Coverage in Dart
Mathieu Kerjouan
Mathieu Kerjouan

Posted on

Tests and Coverage in Dart

#dart #testing #unit #coverage

Writing tests is one of the most important and tedious task a developer has to do. A test is more than proving something is working in a project - even if it's main goal, it's also a way to see how an API is working, by offering implicit use cases and representing data using different format (mocking).

It's usually a good practice to create the test first and then implement the solution. This is the foundation of both Test-driven Development (TDD) and Extreme Programming (XP) methodologies.

In TDD, the test must be written first, and must fail. During the next iteration, the developer will invest time and effort to ensure the test is passing. In XP, this is practically the same, except this methodology has been created for high speed development (game production), and the tests are also used as source of trust and documentation.

This post is not really about those methodologies, even if we can talk a bit about them, one will find more accurate resources on Wikipedia or elsewhere than reading this publication. The goal of this article is to summarize the different ways available to test an application in Dart. Dart only, not Flutter, this is a different beast, and a dedicated article will be created only for it.

Bootstrapping

As usual, a new dedicated project will be created, let call it... rps for rock-paper-scissors. Let say it will be a library containing the rules for the Jan-ken-pon.

$ dart create rps
Creating rps using template console...

  .gitignore
  analysis_options.yaml
  CHANGELOG.md
  pubspec.yaml
  README.md
  bin/rps.dart
  lib/rps.dart
  test/rps_test.dart

Running pub get...                     0.3s
  Resolving dependencies...
  Downloading packages...
  Changed 48 dependencies!

Created project rps in rps! In order to get started, run the following commands:

  cd rps
  dart run
Enter fullscreen mode Exit fullscreen mode

If no template is being used to create this application, one will need to add the test package as dependencies in pubspec.yaml file, inside the dev_dependencies section. See below for an example.

dev_dependencies:
  lints: ^6.0.0
  test: ^1.25.6
Enter fullscreen mode Exit fullscreen mode

The test package is the main one used to create test suites in Dart. It is maintained by the Dart team and available by default in the SDK.

Rock-Paper-Scissors Rules

The Jan-ken-pon is a simple - and addictive - game. The goal is to win a duel with someone else using one of the 3 movements also called Shapes: Rock (the hand is fully closed), Paper (the hand is fully opened), and Scissors (the hand is half closed and the fingers are doing a scissors).

  • The Rock wins over Scissors (then the Scissors is losing against Rock);

  • The Scissors wins over Paper (then the Paper is losing against Scissors);

  • The Paper wins over Rock (then the Rock is losing against Paper);

  • If both players are using the same movement, it's a draw and they must start over until one win.

A quick table to summarize the final results:

a vs b rock paper scissors
rock draw loss win
paper win draw loss
scissors loss win draw

Usually, this game is played anywhere and for any reasons (for example for the one who will eat the last chocolate desert in the restaurant). Let call the act to play with someone else a Party. This post will also give me the opportunity to talk about design programming and OOP. Yeah. Marvelous. Finally.

Rock-Paper-Scissors Shapes

To me, a good programming language should give you an idea of an implementation without a lot of effort. When using Object Oriented Programming paradigm, this is never the case. You don't have one or two obvious solutions, but a bunch of potentials valid solutions. Let try to implement our shapes here (rock, paper and scissors).

Here 3 of the implementations I can think of when dealing with this project:

  1. (medium complexity) Like any other OOP language, one will create a first abstract class called Shape, and then will create 3 classes called Rock, Paper and Scissors inheriting from the Shape class;

  2. (low complexity) Because Dart is being used here, one can do something a bit different, using an enumerator as class, and extends it with few methods to compare the defined value between them;

  3. (high complexity) Create a more complex data-structure, based on a ring (linked list) where the first element is linked to the last one, or a directed graph, where the different path can define which shape greater than another one. Those solutions are way more complex, but can give more flexibility and extends the game in the future with new shapes (e.g. Spoke and Snake).

The second option seems to be the easiest one for this kind of publication, so, let implement it. Let begin with something simple, the representation of the shapes using an emoji (utf8/unicode). A test/ directory should already be present in the project, we can add a new test file called shapes_test.dart there.

// file test/shapes_test.dart

import 'dart:convert';
import 'package:test/test.dart';
import 'package:rps/shapes.dart' as shapes show versus;
import 'package:rps/shapes.dart' show Shape;

final utf8Encoder = utf8.encoder;

void shapeRepresentation() {
  group('Shapes Representation', () {
    // see https://unicodeplus.com/U+270A
    test('rock as raised fist unicode', () {
      expect(
        utf8Encoder.convert(Shape.rock.toString()),
        [226, 156, 138]
      );
    });

    // see https://unicodeplus.com/U+1FAF3
    test('paper as palm down hand unicode', () {
      expect(
        utf8Encoder.convert(Shape.paper.toString()),
        [240, 159, 171, 179]
      );
    });

    // see https://unicodeplus.com/U+270C
    test('scissors as victory hand unicode', () {
      expect(
        utf8Encoder.convert(Shape.scissors.toString()),
        [226, 156, 140]
      );
    });
  });
}

void main() {
  shapeRepresentation();
}
Enter fullscreen mode Exit fullscreen mode

Here's an explanation of this code, a new test case is created with the help of the test() function. The first argument is the description of the test as String, and the next argument is an anonymous function containing the code to be executed.

A test case can be added in a test group, in this situation, the group() function can be used. It follows the same principle than the test() function, the first argument is the description of the group as String and the second argument will contain the test cases to be executed.

Then, each test cases is made of functions exposed by the test module, in our case, the expect() function is used to assert a certain value.

$ dart test
00:00 +0 -1: loading test/shapes_test.dart [E]                                                          
  Failed to load "test/shapes_test.dart":
  test/shapes_test.dart:13:29: Error: Undefined name 'Shape'.
          utf8Encoder.convert(Shape.rock.toString()),
                              ^^^^^
  test/shapes_test.dart:21:29: Error: Undefined name 'Shape'.
          utf8Encoder.convert(Shape.paper.toString()),
                              ^^^^^
  test/shapes_test.dart:29:29: Error: Undefined name 'Shape'.
          utf8Encoder.convert(Shape.scissors.toString()),
                              ^^^^^
  test/shapes_test.dart:39:14: Error: Undefined name 'Shape'.
        expect(Shape.rock > Shape.scissors, equals(true));
               ^^^^^
  test/shapes_test.dart:39:27: Error: Undefined name 'Shape'.
        expect(Shape.rock > Shape.scissors, equals(true));

00:00 +0 -1: Some tests failed.
Enter fullscreen mode Exit fullscreen mode

The tests are failing, this is normal, nothing has been created so far. Let fix that by creating a new file called lib/shapes.dart, it will contain our first implementation.

// file lib/shapes.dart

enum Shape {
  rock, paper, scissors;

  String toString() {           
    switch (this) {
      case rock: return "✊";
      case paper: return "🫳";
      case scissors: return  "✌";
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Let re-invoke dart test to see if our implementation is working correctly.

$ dart test
00:00 +3: All tests passed!
Enter fullscreen mode Exit fullscreen mode

It works! Now we need something to compare Shapes between them; for doing that, let playing with the default operators offered by dart, mainly used to compare numbers. Why not using them to compare Shapes?

// file test/shapes_test.dart

void shapeOperators() {
  group('Shapes Operators', () {
    test('rock operators', () {
      expect(Shape.rock > Shape.scissors, equals(true));
      expect(Shape.rock < Shape.paper, equals(true));
      expect(Shape.rock == Shape.rock, equals(true));
    });

    test('scissors operators', () {
      expect(Shape.scissors > Shape.paper, equals(true));
      expect(Shape.scissors < Shape.rock, equals(true));
      expect(Shape.scissors == Shape.scissors, equals(true));
    });

    test('paper operators', () {
      expect(Shape.paper > Shape.rock, equals(true));
      expect(Shape.paper < Shape.scissors, equals(true));
      expect(Shape.paper == Shape.paper, equals(true));
    });
  });
}

void main() {
  shapeRepresentation();
  shapeOperators();
}
Enter fullscreen mode Exit fullscreen mode

As previously said, the test cases are grouped together. The code here describes how the Shapes will interact when using > (greater than) and < (less than) operators. The code is straightforward and explicit, but one could have done probably more tests with the help of the huge amount of function helpers offered by the test package, here a short list of them:

  • isFalse constant, ensure the final result is returning false as boolean;

  • isTrue constant, same than isFalse except it ensure the returned value is true;

  • isException() contant can be used to be sure the function will throw an exception;

  • contains() function checks if the returned values contain a specific objects;

  • fail() function throws a failure;

  • isA() function checks the type of the data returned;

  • same() function checks if 2 items are identical.

Anyway, let implement these new requested features to compare Shapes using our standard operators. To do that, we will create methods marked with the operator identifier. From the specification:

Operators are instance methods with special names, except for operator [] which is an instance getter and operator []= which is an instance setter. An operator declaration is identified using the built-in identifier operator.

-- Dart Programming Language Specification, Chapter 10, Section 2.1, page 37

In our case, the methods < and > will be created. They must return a boolean.

// file lib/shapes.dart

enum Shape {
  rock, paper, scissors;

  String toString() {           
    switch (this) {
      case rock: return "✊";
      case paper: return "🫳";
      case scissors: return  "✌";
    }
  }

  bool operator <(Shape other) {
    switch ((this, other)) {
      case (rock, paper): return true;
      case (rock, scissors): return false;
      case (paper, rock): return false;
      case (paper, scissors): return true;
      case (scissors, paper): return true;
      case (scissors, rock): return false;
      case (rock,rock): return false;
      case (paper,paper): return false;
      case (scissors,scissors): return false;
    }
  }

  bool operator >(Shape other) {
    switch ((this, other)) {
      case (rock, paper): return false;
      case (rock, scissors): return true;
      case (paper, rock): return true;
      case (paper, scissors): return false;
      case (scissors, paper): return true;
      case (scissors, rock): return false;
      case (rock,rock): return false;
      case (paper,paper): return false;
      case (scissors,scissors): return false;
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Now the implementation is done, let runs the tests.

$ dart test
Building package executable... 
Built test:test.
00:00 +6: All tests passed!
Enter fullscreen mode Exit fullscreen mode

Great, finally, we need an easy way to know if a shape versus another can be defined as a Win, a Loss or a Draw. It can be implemented as an exposed function from the module Shapes directly, we don't need something smart, just a function doing a comparison with two Shape.

// file test/shapes_test.dart

import 'package:rps/results.dart';

void shapeVersus() {
  print(shapes.versus(Shape.rock, Shape.scissors));
  group('Shapes Versus', () {
    test('rock', () {
      expect(
        shapes.versus(Shape.rock, Shape.scissors).toString(),
        equals("Win")
      );
      expect(
        shapes.versus(Shape.rock, Shape.rock).toString(),
        equals("Draw")
      );
      expect(
        shapes.versus(Shape.rock, Shape.paper).toString(),
        equals("Loss")
      );
    });
  });
}
Enter fullscreen mode Exit fullscreen mode

New problem. How to represent a Draw, a Win and a Loss? Well, all can be seen as Result, so, let create at first an abstract class called Result first. Then, we can create the needed classes. This code will be stored in lib/results.dart.

// file lib/results.dart

abstract class Result {}

class Draw extends Result {
  toString() => "Draw";
}

class Win extends Result {
  toString() => "Win";
}

class Loss extends Result {
  toString() => "Loss";
}
Enter fullscreen mode Exit fullscreen mode

Great, let work on the versus() function, we can implement it in many ways, let implement the easiest one, using the operators previously created.

// file lib/shapes.dart

Result versus(Shape left, Shape right) {
  if (left == right) return Draw();
  if (left > right) return Win();
  if (left < right) return Loss();
  throw("error");
}
Enter fullscreen mode Exit fullscreen mode

Now our implementation is done, let check the tests.

$ dart test
00:00 +7: All tests passed!
Enter fullscreen mode Exit fullscreen mode

Now we have a quick and dirty implementation of the different available Shapes in Jan Ken Pon, let have a look on the test package and all the features it offers us.

Test Reports

The test package offers a way to create test reports using reporters. The default reporter used is the compact one, only printing the sum of successful test or printing the errors. Another simple reporter is the expanded one, it will display more information about the test suite.

$ dart test --reporter=expanded
00:00 +0: loading test/rps_test.dart
00:00 +0: test/shapes_test.dart: Shapes Representation rock as raised fist unicode
00:00 +1: test/shapes_test.dart: Shapes Representation paper as palm down hand unicode
00:00 +2: test/shapes_test.dart: Shapes Representation scissors as victory hand unicode
00:00 +3: test/shapes_test.dart: Shapes Operators rock operators
00:00 +4: test/shapes_test.dart: Shapes Operators scissors operators
00:00 +5: test/shapes_test.dart: Shapes Operators paper operators
00:00 +6: test/shapes_test.dart: Shapes Versus rock
Enter fullscreen mode Exit fullscreen mode

If your project is running on Github and you are using Github Actions, a specific reporter called github can be used and will output a Github friendly report.

$ dart test --reporter=github

::group::✅ Passing tests
✅ test/shapes_test.dart: Shapes Representation rock as raised fist unicode
✅ test/shapes_test.dart: Shapes Representation paper as palm down hand unicode
✅ test/shapes_test.dart: Shapes Representation scissors as victory hand unicode
✅ test/shapes_test.dart: Shapes Operators rock operators
✅ test/shapes_test.dart: Shapes Operators scissors operators
✅ test/shapes_test.dart: Shapes Operators paper operators
✅ test/shapes_test.dart: Shapes Versus rock
::endgroup::

🎉 7 tests passed.
Enter fullscreen mode Exit fullscreen mode

Finally, if you are using your own CI/CD pipeline with Jenkins or Robot Framework, the result of the test can be returned as JSON with the json reporter.

$ dart test --reporter=json
{"protocolVersion":"0.1.1","runnerVersion":"1.31.1","pid":3420883,"type":"start","time":0}
{"suite":{"id":0,"platform":"vm","path":"test/rps_test.dart"},"type":"suite","time":0}
{"test":{"id":1,"name":"loading test/rps_test.dart","suiteID":0,"groupIDs":[],"metadata":{"skip":false,"skipReason":null},"line":null,"column":null,"url":null},"type":"testStart","time":0}
...
{"success":true,"type":"done","time":310}
Enter fullscreen mode Exit fullscreen mode

Test Coverage

Generating a coverage report via dart test requires LCOV tool installed on your system. If you are using a Debian-like distribution, the LCOV package should be available.

$ sudo apt-get install lcov
Enter fullscreen mode Exit fullscreen mode

To generate a new coverage report, simply use --coverage-path arguments. It will create the directory destination if it does not exist and generate the report.

$ dart run test --coverage-path=./_coverage/lcov.info
00:00 +7: All tests passed!
Enter fullscreen mode Exit fullscreen mode

If you want to have an HTML report, the genhtml command usually present with the LCOV package is needed. It will generate an HTML report in the same directory the lcov.info file has been created.

$ genhtml -o _coverage/report ./_coverage/lcov.info 
Reading tracefile ./_coverage/lcov.info.
Found 3 entries.
Found common filename prefix "/dart/rps"
Generating output.
Processing file lib/rps.dart
  lines=2 hit=0
Processing file lib/results.dart
  lines=3 hit=3
Processing file lib/shapes.dart
  lines=28 hit=18
Overall coverage rate:
  source files: 3
  lines.......: 63.6% (21 of 33 lines)
  functions...: no data found
Message summary:
  no messages were reported
Enter fullscreen mode Exit fullscreen mode

Now, the final result. See below the 3 levels of coverage from the top layer (a global view of the application) and the 2 other layers. The third one is the coverage from the module directly, indicating the numbers of time a line is called.

Top Level Coverage Report

Lib Coverage Report

Shapes Module Coverage Report

Conclusion

Testing an application with Dart is an easy task, the tools offered by default are doing what any developer is looking for, unit testing, test coverage and reports. It was a really quick introduction to this complex topic, we still have a lot of things to learn, like mocking techniques, test integration, testing frontend, backend or flutter and much more.

The skeleton of this project can be found in niamtokik/rps_dart repository on Github. If you are still there and want to know more, here a list of interesting resources:

Have fun!

Cover Image by Tom Crew on Unsplash

Top comments (0)