Testing PHP Applications

This article was originally written by
Mauro Chojrin on the Honeybadger Developer Blog.

Lately, there has been a lot of movement around new and exciting php testing tools. While this is great news, stepping back a little and understanding the underlying concepts before jumping in would be helpful. Let's start by agreeing that when we talk about testing tools and methodologies, we're referring to automated testing tools.

What is Automated Testing?

Automated testing refers to using software tools designed specifically to run an application with a particular set of inputs and check the produced output against a known set of expectations. In essence, it's not very different from regular (i.e., manual) testing. It's just way better.

Why is Automated Testing a Good Idea?

Automated testing helps to significantly reduce the bug count and increases the quality of software under development. Similarly, when applied to established applications, it helps prevent the appearance of regression bugs, which refers to the re-introduction of fixed errors when adding new features/improvements to working code.

You may be skeptical if you have never used automated testing tools. After all, if the testing tools are software, how do you keep them from having bugs? Fair enough. The trick here is to write the tests in a way that, should they contain a bug, it would be trivial to find and fix. Usually, this means writing small tests and combining them into test suites.

The bottom line is that automated testing can seem like a waste of time at first, as it consumes many resources at the beginning of a project, but the long-term payoff is absolutely worth it. Trust me on this one.

What Types of Automated Testing exist?

Now that we've covered the basic concepts, let's go one step further. As you may have already guessed (or heard somewhere), there are several types of automated tests. The main difference is what exactly is being tested. You can think of it as how close to the code you want the lens to be. On the most extreme end, you find unit tests, and the furthest you can go while staying relevant is some flavor of acceptance testing.

Another way to categorize the tests is by how much knowledge about the system being tested you have. In this scheme, you'll find what’s referred to as white-box vs. black-box testing. In the first case, you have access to and the ability to understand the code, while in the latter, the opposite is true.

Notably, the different kinds of tests are not necessarily mutually exclusive. In fact, in most cases, the more layers of tests you add to your projects, the better.

Of course, if you go overboard, you can find yourself spending way more time writing tests than code that actually solves business problems ... but that's a discussion for a different article.

In the following sections, I'll show you how to implement some specific tools in a somewhat real project. These examples have been built and tested against php 8.0 on an Ubuntu box, and I used Google Chrome as a Web browser. If your setup doesn't exactly match these specifications, you may need to tweak the commands a little.

Unit Tests

Let's start at the very basic level: unit tests. In this kind of testing, you're trying to determine whether a particular unit of code complies with a set of expectations.

For instance, if you were creating a calculator class, you'd expect to find the following methods in it:

• add
• subtract
• multiply
• divide

In the case of the add method, the expectation is rather clear; it should return the result of adding two numbers.

Therefore, our class should look like this:

declare(strict_types=1);

class Calculator
{
    public function add(int $a, int $b): int
    {
        return $a + $b;
    }
}

There are many tools you can use to prove the correct working of this method. The most popular one (by far!) is phpUnit.

The first thing you should do to add it to your project is install the tool. The best way to go about it is to add it as a dependency of the project. Assuming that your project is built using composer, all you need to do is issue a composer require --dev phpunit/phpunit, which will produce a new file inside your vendor/bin directory: phpunit.

So, before going any further, make sure everything is in place. Run vendor/bin/phpunit --version. If everything goes well, you should see something like this:

PHPUnit 9.5.10 by Sebastian Bergmann and contributors.

Great! You're ready to test your code!

Of course ... that's not saying much if you haven't written any tests, right?

So, how do you write your first unit test using phpUnit?

Start by creating a new directory called tests at the root of your project.

In it, create a file named CalculatorTest.php and put the following inside of it:

<?php

declare(strict_types=1);

use PHPUnit\Framework\TestCase;

final class CalculatorTest extends TestCase
{
        public function testAddAdds()
        {
                $sut = new Calculator();
                $this->assertEquals(8, $sut->add(5, 3));
        }
}

Before running the test, a few things need to be in place:

1. A phpUnit configuration file (phpunit.xml.dist) at the project root.
2. A bootstrap script to bring autoloading in.
3. An autoload definition.

Don't worry about this part; it sounds much worse than it actually is.

The phpUnit config is just a shortcut to avoid explicitly feeding options every time you run the phpunit command. In our case, a simple one will do, like this:

<phpunit
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/3.7/phpunit.xsd"
        backupGlobals="true"
        backupStaticAttributes="false"
        bootstrap="tests/bootstrap.php">
</phpunit>

The most important part of this file is the definition bootstrap="tests/bootstrap.php", which establishes tests/bootstrap.php as the entry point to our test suite, hence the need to create such file.

The contents of tests/bootstrap.php don't need to be very elaborate either. This will do just fine:

<?php

require_once __DIR__.'/../vendor/autoload.php';

Finally, we need to inform composer about our class mapping to allow autoloading to be successful. Simply add the following to your composer.json:

  "autoload": {
        "psr-4": {
            "" : "."
        }
    },

Then, run composer dump-autoload to generate the file vendor/autoload.php, and you'll be ready to run your tests without surprises.

Issue the command vendor/bin/phpunit tests at the root of your project, and you'll see something like:

PHPUnit 9.5.10 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 00:00.003, Memory: 4.00 MB
OK (1 test, 1 assertion)

What this means is that the assertion (the fact that 8 equals the result of Calculator::add(5, 3)) was verified during the test run.

There are MANY nuances to phpUnit. In fact, whole books have been written on the subject, and the idea of this article is not to address one particular tool but rather to give you an overview, so you can read further about the ones you find interesting. Still, one little nugget I find very interesting about phpUnit, and I hope it will make you curious. Look at what happens if you run your test using ./vendor/bin/phpunit tests --testdox:

PHPUnit 9.5.10 by Sebastian Bergmann and contributors.

Calculator
 ✔ Add adds

Time: 00:00.003, Memory: 4.00 MB

OK (1 test, 1 assertion)

Not bad, right? But ... where did this text come from? It was taken straight from the test method name so ... mind your tests names!

Integration Tests

The next step in our journey is integration tests. These tests are meant to prove how well some components play together with others.

At first, this type of testing may seem superfluous. After all, if every individual unit does its job, why would you need more tests? Well ... let me give you a graphic explanation:

You definitely don't want to find yourself in this situation.

It may come as a surprise, but despite of its name, phpUnit can also be used to write this kind of test.

In our example, let's assume we will have another component besides our little Calculator, perhaps a numbers Collection that will be able to calculate the sum of its members by feeding them to our Calculator.

It would look something like this:

<?php

declare(strict_types=1);

class NumberCollection
{
    private array $numbers;
    private Calculator $calculator;

    public function __construct(array $numbers, Calculator $calculator)
    {
        $this->numbers = $numbers;
        $this->calculator = $calculator;
    }

    public function sum() : int
    {
        $acum = 0;

        foreach ($this->numbers as $number) {
            $acum = $this->calculator->add($acum, $number);
        }

        return $acum;
    }
}

In this example, you can see how the Calculator is being injected into the NumberCollection. While we could have written the code in to have the constructor create the Calculator instance, it would have made our tests, especially unit tests, much harder to write, among other problems.

In fact, to have a really solid structure, we should be using a CalculatorInterface as the constructor parameter, but we’ll leave this for a different discussion.

I'll leave the unit tests for this class as homework for you and move right into the integration test. In such a test, what I want to determine is whether the two classes work together and eventually produce the result I'm looking for.

How will I do that? Well ... not very differently from what I've done so far. This is what the test will look like:

<?php

use PHPUnit\Framework\TestCase;

class NumberCollectionTest extends TestCase
{
    public function testSum()
    {
        $numbersList = [6, 5, 6, 9];
        $numberCollection = new NumberCollection($numbersList, new Calculator());

        $this->assertEquals(array_sum($numbersList), $numberCollection->sum(), 'Sum doesn\'t match');
    }
}

And then, by running vendor/bin/phpunit tests/NumberCollectionTest.php I get the following:

PHPUnit 9.5.10 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 00:00.003, Memory: 4.00 MB

OK (1 test, 1 assertion)

The difference between this and a unit test is that in the latter, I'd use a mock instead of the actual Calculator class since what I want to test is just the NumberCollection, thus assuming the Calculator works as expected.

Acceptance Tests

Another category of tests you can use is acceptance tests. These tests are meant to be performed by non-technical people, which means that a user is simply pushing buttons and smiling at the results (or crying or yelling, you never know).

This kind of test would clearly not be very repeatable, much less efficient, right? Well, the purpose of this kind of test is to simulate what a real user would do.

In the case of a PHP application, chances are that we're talking about a web application. Therefore, to test it, a Web browser would certainly come in handy.

There are many tools you can use for this purpose, but one I particularly like is CodeCeption. What I like most about it is that it's a unified tool that can be used to perform several types of tests, acceptance being one of them.

Let's start by bringing it into our project, shall we?

Start by running composer require "codeception/codeception" --dev. This will download and install all the required libraries inside your vendor directory.

Next, initialize CodeCeption's environment with the command php vendor/bin/codecept bootstrap, which should produce the following output:

❯ php vendor/bin/codecept bootstrap
 Bootstrapping Codeception 

File codeception.yml created       <- global configuration
 Adding codeception/module-phpbrowser for PhpBrowser to composer.json
 Adding codeception/module-asserts for Asserts to composer.json
2 new packages added to require-dev
? composer.json updated. Do you want to run "composer update"? (y/n)

Answer y and wait for it to complete downloading all the auxiliary packages.

Now, there's a lot to do here to really take advantage of CodeCeption. For the moment, let's focus on putting together an acceptance test. To do this, we'll need an application that can be tested via a browser.

Let's go back to our little Calculator and add a web UI to it.

Create a web directory at the root of your project and put the following code inside an index.php file:

<?php
require_once '../vendor/autoload.php';

session_start();

if ('post' === strtolower($_SERVER['REQUEST_METHOD'])) {
    $_SESSION['numbers'][] = (int)$_POST['newNumber'];
}

$numbers = $_SESSION['numbers'] ?? [];
$numbersCollection = new NumberCollection($numbers, new Calculator());
?>
<html>
<body>
    <p>Numbers entered: <b><?php echo implode(', ', $numbers); ?></b></p>
    <p>Sum: <b><?php echo $numbersCollection->sum();?></b></p>
    <hr/>
    <form method="post">
        <label for="newNumber">Enter a number between 1 and 100:</label>
        <input type="number" min="1" max="100" name="newNumber" id="newNumber"/>
        <input type="submit" value="Add it!"/>
    </form>
</body>
</html>

Now, run the built-in web browser by running php -S localhost:8000 -t web, and voilá, you’ve got a nice web UI at http://localhost:8000, which should look similar to the following:

Now that we have everything in place let's go return our original goal: put together an acceptance test.

To do this, we'll need to tweak the default configuration a bit.

Open the file tests/acceptance.suite.yml and edit it to look like this:

# Codeception Test Suite Configuration
#
# Suite for acceptance tests.
# Perform tests in browser using the WebDriver or PhpBrowser.
# If you need both WebDriver and PHPBrowser tests - create a separate suite.

actor: AcceptanceTester
modules:
    enabled:
        - WebDriver:
            url: http://localhost:8000
            browser: chrome
        - \Helper\Acceptance
step_decorators: ~        

Here, we're asking CodeCeption to run our tests in an actual Web browser, so we will need the support of a couple of auxiliary tools:

1. CodeCeption's WebDriver module
2. ChromeDriver

To install the WebDriver module, simply run composer require codeception/module-webdriver --dev.

To install ChromeDriver, first check your Chrome version by going to Help -> About Chrome. Once you know the exact version number you have installed, go here and download the version that matches your installation.

When ready, run ./chromedriver --url-base=/wd/hub --white-list-ip 127.0.0.1 to init the Chrome driver server.

I know, I know ... it seems like too much work just to run a few tests, right? Well, it might be, but remember that this is something you'll do just once, and then it'll make your applications so much easier to test and, thus, much more reliable.

It’s time to see some actual php code, isn't it? Let's go straight to it!

Use this command to create your first CodeCeption-based acceptance test: vendor/bin/codecept g:cest acceptance First, and then open the file tests/acceptance/FirstCest.php to find the following:

<?php

class FirstCest
{
    public function _before(AcceptanceTester $I)
    {
    }

    // tests
    public function tryToTest(AcceptanceTester $I)
    {
    }
}

It doesn't look like much, does it? Bear with me for a minute; the magic is about to begin.

One thing we might want to test in this scenario is the user's ability to enter a number and r the expected result, so let's write this exact test.

Edit the method tryToTest of the FirstCest class to look like this:

    public function tryToTest(AcceptanceTester $I)
    {
        $I->amOnPage('index.php');
        $I->amGoingTo('put a new number into the collection');
        $I->see('Numbers entered:');
        $I->see('Sum:');
        $newNumber = rand(1, 100);
        $I->fillField('newNumber', $newNumber);
        $I->click('Add it!');
        $I->wait(2);
        $I->see('Numbers entered: '.$newNumber);
        $I->see('Sum: '.$newNumber);
    }

Then, run vendor/bin/codecept run acceptance.

Go ahead; I'll wait here.

Done? Good.

I guess that by now, you'll see why I like CodeCeption so much. If not, take a look back at the test you just wrote. Note how clear and easy it was to write. It’s certainly much cleaner and elegant that its bare phpUnit counterpart, right?

The fact is, behind the scenes CodeCeption uses phpUnit to actually run the tests, but it takes the experience to a whole new level.

Testing Methodologies

In the realm of software testing, there are many approaches to how to write tests, as well as when to write them.

Let's have a quick look at two of the most popular ones.

TDD in PHP

The acronym TDD stands for Test Driven Development. The idea here is to write your tests before writing the actual code. Sounds strange, doesn't it? How will you know what to test before the code is written? This is exactly the idea. It's about writing the minimum code needed to pass the tests.

If tests are good enough, the very passing of them should be proof that the code matches its functional requirements and there's no extra code.

In terms of tools, there's not really much to add to what we’ve already discussed. Usually, phpUnit is the preferred tool for these kind of tests; the only thing that changes is the order of execution.

BDD in PHP

BDD certainly is a different way to think about software testing. The idea of BDD is somewhat similar to TDD in the sense that it's based on a cycle of the following

1. Test
2. Write some running code
3. Adjust
4. Go back to 1

However, the way the tests are written is radically different. In fact, tests are supposed to be written in a language that can both be understood by developers and business people (i.e., examples). There is a language designed specifically for this purpose; it's called Gherkin.

Let's look at a quick example of what this would mean in our Calculator app:

Feature: Numbers collection
  In order to calculate the sum of a series of numbers
  As a user
  I need to be able to input numbers

  Rules:
  - Numbers should be integers between 1 and 100

  Scenario: Input the first number
    Given the number series is empty
    When I enter 2
    Then The number series should contain only 2
    And The sum should be 2

  Scenario: Input the second number
    Given the number series contains 5
    When I enter 10
    Then The number series should contain 5 and 10
    And The sum should be 15    

To do something with this definition, we need to bring Behat in. As usual, we'll rely on Composer to help with this task.

Issue the command composer require --dev behat/behat. Then, we need to initialize the test suite with the command vendor/bin/behat --init. This command will create the basic structure needed for Behat to run. The most important part of this is the creation of the features directory, where our feature descriptions will live.

So, naturally, the next step is to take the Gherkin text we wrote and save it to a .feature file. In our case, let's call it number_collection.feature.

Okay, we're ready to get our hands dirty. Run vendor/bin/behat --append-snippets, and you'll see how Behat interprets your feature, recognizing two scenarios and eight steps.

Since this is the first time you’ve run Behat on this project, there's quite a bit of work ahead. After all, the text definition looks great, but when it comes to having a computer check it against reality, I'm afraid AI hasn’t developed that far yet. We're going to have to help it by filling in the blanks.

Ultimately, you should end up with a features/bootstrap/FeatureContext.php file that looks like this:

<?php

use Behat\Behat\Context\Context;
use Behat\Gherkin\Node\PyStringNode;
use Behat\Gherkin\Node\TableNode;

/**
 * Defines application features from the specific context.
 */
class FeatureContext implements Context
{
    /**
     * Initializes context.
     *
     * Every scenario gets its own context instance.
     * You can also pass arbitrary arguments to the
     * context constructor through behat.yml.
     */
    public function __construct()
    {
    }

    /**
     * @Given the number series is empty
     */
    public function theNumberSeriesIsEmpty()
    {
        throw new PendingException();
    }

    /**
     * @When I enter :arg1
     */
    public function iEnter($arg1)
    {
        throw new PendingException();
    }

    /**
     * @Then The number series should contain only :arg1
     */
    public function theNumberSeriesShouldContainOnly($arg1)
    {
        throw new PendingException();
    }

    /**
     * @Then The sum should be :arg1
     */
    public function theSumShouldBe($arg1)
    {
        throw new PendingException();
    }

    /**
     * @Given the number series contains :arg1
     */
    public function theNumberSeriesContains($arg1)
    {
        throw new PendingException();
    }

    /**
     * @Then The number series should contain :arg1 and :arg2
     */
    public function theNumberSeriesShouldContainAnd($arg1, $arg2)
    {
        throw new PendingException();
    }
}

Take a minute to go over this file.

You should note that there's a clear mapping between the textual definition you created using Gherkin and the method names created by Behat. Of course, this is no coincidence. Furthermore, take a look at the annotations above the method names. What is happening there is the declaration of the precise mapping between the Gherkin definitions and the code that will make them executable.

Looks nice, doesn't it?

There's just a little problem. This code, by itself, doesn't really do much. What's missing here is the setting of the context for test execution. Basically, you have to initialize the objects needed for later testing at the methods identified by the @Given annotation, the changes made to them in those methods annotated with @When, and finally, the assertions needed to validate the expectations expressed by the @Then annotations.

Let's look at the complete example for clarity:

<?php

declare(strict_types=1);
use Behat\Behat\Context\Context;
use Behat\Gherkin\Node\PyStringNode;
use Behat\Gherkin\Node\TableNode;
use NumberCollection;
use PHPUnit\Framework\Assert;

/**
 * Defines application features from the specific context.
 */
class FeatureContext implements Context
{
    private NumberCollection $numberCollection;

    /**
     * Initializes context.
     *
     * Every scenario gets its own context instance.
     * You can also pass arbitrary arguments to the
     * context constructor through behat.yml.
     */
    public function __construct()
    {
        $this->numberCollection = new NumberCollection([], new Calculator());
    }

    /**
     * @Given the number series is empty
     */
    public function theNumberSeriesIsEmpty()
    {
    }

    /**
     * @When I enter :arg1
     */
    public function iEnter(int $arg1)
    {
        $this->numberCollection->append($arg1);
    }

    /**
     * @Then The number series should contain only :arg1
     */
    public function theNumberSeriesShouldContainOnly(int $arg1)
    {
        $numbers = $this->numberCollection->getNumbers();
        Assert::assertContains($arg1, $numbers);
        Assert::assertCount(1, $numbers);
    }

    /**
     * @Then The sum should be :arg1
     */
    public function theSumShouldBe(int $arg1)
    {
        Assert::assertEquals($arg1, $this->numberCollection->sum());
    }

    /**
     * @Given the number series contains :arg1
     */
    public function theNumberSeriesContains(int $arg1)
    {
        $this->numberCollection->append($arg1);
    }

    /**
     * @Then The number series should contain :arg1 and :arg2
     */
    public function theNumberSeriesShouldContainAnd(int $arg1, int $arg2)
    {
        Assert::assertContains($arg1, $this->numberCollection->getNumbers());
        Assert::assertContains($arg2, $this->numberCollection->getNumbers());
    }
}

Don't get confused by the fact that phpUnit assertions are being used here; any other assertion library would work just as well.

Behat is great, but it's not the only one. In fact, CodeCeption also features Gherkin support.

Some Other Interesting PHP-Based Testing Tools

In this section, I'll make quick mention of some tools I haven't had the chance to try out myself but look promising:

• Infection: a mutation testing tool
• Pest: a testing framework designed following Laravel coding standards
• Atoum: a simple PHP testing framework
• phpSpec: another BDD framework for PHP

You can get the full example from GitHub if you want to check it out.

Wrapping Up

As you can see, there's a lot going on in the PHP testing arena. Furthermore, there are many people working on pushing the limits when it comes to software quality.

If you're not yet using any of these wonderful tools, now is the time to start. Moreover, while you're at it, it'd be great if you could also pick up a static analysis one.

The time for professional PHP developers has come; don't get left behind.

Comments

[hubertinio]

There is wrong link for Atoum