DEV Community: Filmin Engineering

Stopping time with PHP

Rubén Rubio — Tue, 09 Nov 2021 08:16:16 +0000

Introduction

It is a good practice to use an interface to manage the clock in an application, as it allows having full control of time. For example, it eases testing, as it lets us define the concrete time for each test. Frank de Jonge and Matthias Noback have blog posts about it, brick has an implementation, and there is even a PSR proposal to have a ClockInterface.

However, there are more use cases for ClockInterface, besides testing. In our case, we had to implement endpoints to show the movies that are just released, and the ones that are about to release in the next few days or weeks.

Being an API, it is easy to test using PHPUnit, as you could easily set the required date for each test. However, our frontend team had to integrate the website (a Vue app) with this endpoint. This feature is easy to develop and test locally if you have a fresh backup of the production database, as the date of your local system will be in sync with the data within your local database. However, what if you do not have your local database updated, and getting a copy from production takes too much time?

For sure, an option would be to update the local data either by querying directly the database or by using a back-office, if there is one. Nonetheless, this is a manual task that could take some time if you have to update many elements. And it is something that you have to do recurrently if you need to review the implementation after the date you set up passes.

But there is another option: we can take advantage of the ClockInterface to fixate the date of the whole system.

Implementation

Having a ClockInterface such as:



interface ClockInterface
{
    public function now(): DateTimeImmutable;
}

We could have this FixedClock implementation:



use DateTimeImmutable;
use DateTimeZone;
use Exception;
use InvalidArgumentException;

final class FixedClock implements ClockInterface
{
    public const FORMAT = 'Y-m-d H:i:s';

    private ?DateTimeImmutable $dateTime;
    private DateTimeZone $timeZone;

    public function __construct(string $timeZone, string $dateTime = '')
    {
        $this->timeZone = $this->parseAndValidateTimeZone($timeZone);
        $this->dateTime = $this->parseAndValidateDateTime($dateTime);
    }

    public function now(): DateTimeImmutable
    {
        if ($this->dateTime !== null) {
            return $this->dateTime;
        }

        return new DateTimeImmutable('now', $this->timeZone);
    }

    private function parseAndValidateTimeZone(string $timeZone): DateTimeZone
    {
        try {
            return new DateTimeZone($timeZone);
        } catch (Exception) {
            throw new InvalidArgumentException(
                sprintf('Value "%s" for time zone is not valid.', $timeZone)
            );
        }
    }

    private function parseAndValidateDateTime(string $dateTime): ?DateTimeImmutable
    {
        $parsedDateTime = trim($dateTime);

        if ($parsedDateTime === '') {
            return null;
        }

        $dateTimeImmutable = DateTimeImmutable::createFromFormat(
            self::FORMAT,
            $parsedDateTime,
            $this->timeZone
        );

        if (! $dateTimeImmutable) {
            throw new InvalidArgumentException(
                sprintf(
                    'Value "%s" for date time is not valid. Expected to have the format "%s"',
                    $dateTime,
                    self::FORMAT
                )
            );
        }

        return $dateTimeImmutable;
    }
}

It is not a good practice to perform anything besides assign properties in the constructor, but it will prove useful afterwards.

Configuration

Symfony

We only need to set the fixed clock up for our development environment. By default, Symfony loads the services.yaml file and then the services_{environment.yaml}, so you can override any service on per-environment basis. To do so, we add to our services_dev.yaml the following lines:



parameters:
    timezone: '%env(string:TIMEZONE)%'
    now: '%env(string:NOW)%'

services:
    _defaults:
        autowire: true
        autoconfigure: true

    Filmin\SharedKernel\Domain\ValueObject\Time\FixedClock:
        arguments:
            - '%timezone%'
            - '%now%'

    Filmin\SharedKernel\Domain\ValueObject\Time\ClockInterface: '@Filmin\SharedKernel\Domain\ValueObject\Time\FixedClock'

What we do here is:

Lines 1-3: set up container parameters that contain the timezone and the current time from environment variables.
Lines 9-12: setup our FixedClock implementation as a service. In Symfony, we can not use named constructors when defining value objects as services. Thereby, we validated the values within the constructor, as shown above.
Line 14: alias the ClockInterface to our FixedClock, so any service that type hint a ClockInterface will use the FixedClock implementation, thus overriding any previous, generic definition.

If we do not set up any value for the environment variable NOW, the application will use the system time. Therefore, switching between a fixed time and the system time is as easy as to update an environment variable.

Laravel

In a Laravel application, we would configure the ClockInterface within a service provider using a singleton, such as:



$this->app->singleton(ClockInterface::class, function (Application $app) {
    if ($app->environment('local')) {
        return new FixedClock(getenv('TIMEZONE'), getenv('NOW'));
    }

    return new SystemClock();
});

In this case, SystemClock is an implementation of ClockInterface that uses the system time, that is what we want in production. As with the previous case, switching between the system time and a fixed time is as easy as to update an environment variable.

Note that we are not experts in Laravel, so this configuration could probably be improved.

Conclusion

We saw that using a ClockInterface eases the testing process and allow fixating the time of the whole application, that can be useful for local testing purposes.

If you are implementing a new application, it is simple to start using ClockInterface everywhere. However, if you are working on legacy code, it could take some time until you have the ClockInterface in your whole application. In those cases, it could be useful to apply the boy scout rule: "Always leave the code better than you found it".

PHP libraries and tools

Rubén Rubio — Wed, 22 Sep 2021 10:10:25 +0000

Introduction

We want the new project we are starting at Filmin to be robust. Therefore, besides adopting good coding practices, we checked some PHP libraries and tools that can help us improve its quality.

Here is a list of the tools we analyzed with the description they have in its Github repository, excluding well known tools or libraries, such as PHPUnit.

Most of these tools are CLI commands, so they can be integrated in a CI/CD flow. Others can be integrated within the IDE, such as Psalm or PHPStan in PHPStorm. And others are libraries that we use in our project.

The list is based in our experience, and it is heavily influenced by:

Libraries

brick/date-time: Date and time library for PHP
thecodingmachine/safe: All PHP functions, rewritten to throw exceptions instead of returning false, now for php8
Tactician: A small, flexible command bus
ronanguilloux/isocodes: PHP library - Validators for standards from ISO, International Finance, Public Administrations, GS1, Manufacturing Industry, Phone numbers & Zipcodes for many countries
box/spout: Read and write spreadsheet files (CSV, XLSX and ODS), in a fast and scalable way
league/flysystem: Abstraction for local and remote filesystems
Serializer:
- Symfony Serializer Component: The Serializer component is meant to be used to turn objects into a specific format (XML, JSON, YAML, …) and the other way around.
- JMS Serializer: This library allows you to (de-)serialize data of any complexity. Currently, it supports XML and JSON.
- Fractal: Fractal provides a presentation and transformation layer for complex data output, the like found in RESTful APIs, and works really well with JSON. Think of this as a view layer for your JSON/YAML/etc.

Tools

Code quality

Deptrac: Keep your architecture clean.
PHP Insights: Instant PHP quality checks from your console
GrumPHP: A PHP code-quality tool.
churn-php: Discover files in need of refactoring.
Psalm: A static analysis tool for finding errors in PHP applications. Plugins:
- boesing/psalm-plugin-stringf: Psalm plugin to provide more details for sprintf, printf, sscanf and fscanf functions.
- hectorj/safe-php-psalm-plugin: vimeo/psalm plugin for thecodingmachine/safe.
- marartner/psalm-no-empty: Psalm plugin to detect usage of empty().
- marartner/psalm-strict-equality: Psalm plugin to enforce strict equality.
- psalm/plugin-phpunit: A PHPUnit plugin for Psalm.
- psalm/plugin-symfony: Psalm Plugin for Symfony.
- weirdan/doctrine-psalm-plugin: Stubs to let Psalm understand Doctrine better.
PHPStan: PHP Static Analysis Tool - discover bugs in your code without running it!. Plugins:
- ergebnis/phpstan-rules: Provides additional rules for phpstan/phpstan.
- spaze/phpstan-disallowed-calls: PHPStan rules to detect disallowed calls and constant & namespace usages
- roave/no-floaters: static analysis rules to prevent IEEE-754 floating point errors.
- More extensions

Testing

Infection: PHP Mutation Testing library. Plugins:
- roave/infection-static-analysis-plugin: Static analysis on top of mutation testing - prevents escaped mutants from being invalid according to static analysis
- bitexpert/captainhook-infection: Captain Hook Plugin to run InfectionPHP only against the changed files of a commit
roave/no-leaks: PHPUnit Plugin for detecting Memory Leaks in code and tests
lulco/populator: Allows populate fake data to your database.
OpenAPI PSR-7 Message (HTTP Request/Response) Validator: It validates PSR-7 messages (HTTP request/response) against OpenAPI specifications.
Paratest: Parallel testing for PHPUnit

Composer tools

ComposerRequireChecker: A CLI tool to check whether a specific composer package uses imported symbols that aren't part of its direct composer dependencies
composer-unused: Show unused composer dependencies by scanning your code
composer-normalize: Provides a composer plugin for normalizing composer.json.

Security

roave/security-advisories: Security advisories as a simple composer exclusion list, updated daily
roave/backward-compatibility-check: Tool to compare two revisions of a class API to check for BC breaks
Local PHP Security Checker: PHP security vulnerabilities checker

General

Robo: Modern Task Runner for PHP
CaptainHook: Very flexible git hook manager for php developers

Conclusion

There is a huge amount of tools written by the PHP community that helps us increase the quality of our project, and that ease our task as developers. We need to choose the ones that adapt better to our project and use case.

And you? Do you know any interesting tool that is not in this list?

Testing strategies

Rubén Rubio — Wed, 08 Sep 2021 10:30:25 +0000

Introduction

We are starting a new project at Filmin, so we are having some discussions to take decisions. We want tests to be one of the cornerstones of this project, so we needed to decide how to implement them.

However, we have not found yet a clear, unique definition of the different kind of testing strategies. What is the scope of an integration test? Should it also test the whole framework? What is the difference between acceptance tests and end-to-end tests?

As we did not have any unique guide, what we did was to have a dialogue between all the team members, interchanging our expertise and viewpoints, to get to a consensus.

These are the testing strategies we agreed on. Note that it is not meant to be an exhaustive list, neither an academic research: these are testing strategies that best suits us for the new project.

Testing strategies

We are using hexagonal architecture, so for each kind of test we will write which layers it applies to. Besides, as we are using PHP, we will write the tool we will use for that testing strategy.

Unit

Unit tests check every part of the software in isolation. Unit tests do not go beyond its scope (i.e., memory) to run,
so they use stubs for interfaces defined within the Domain Layer. Their execution must be fast.

Who needs to understand them: developer.
Apply to:
- Domain layer
- Application layer
Tool: PHPUnit

Integration

Integration tests check that a concrete implementation of a
Domain interface works as expected —but it is not only restricted to elements in Domain.
An example could be a query to the database to check if filtering or pagination work as expected.

They usually require external infrastructure in order to run, such as a Database or an AMQP instance. Locally, these
tests will run against the local Docker instance. In CI/CD, the pipeline will prepare the required infrastructure using
and building a Docker image.

Usually we will need fixtures for the database, i.e., elements in it that we can query or act on. In that case, we will
load the fixtures querying the database directly, loading the bunch of SQL files with queries that we will need. Each test case will be wrapped in a transaction, so they will be isolated.

Who needs to understand them: developer.
Apply to:
- Infrastructure layer
Tool: PHPUnit + Docker with all infrastructure elements needed.

Functional tests

Functional tests are a "unitary" test of the whole stack (including the framework). They should check that the whole
layers (Domain -> Application -> Infrastructure) work properly together. They should check that the expected changes
in infrastructure are correctly done, for example, that a row is inserted in the database or that a message is sent to
Rabbit.

Functional tests also check the response: its status code, its possible responses —including errors—, its possible
formats —JSON, XML, Proto...

Like integration tests, functional tests require infrastructure to run. We will follow the same strategy as with
integration tests.

As integration tests, functional tests also require loading fixtures to the database previously. We will follow the
same strategy as with integration tests.

Who needs to understand them: developer.
Apply to:
- The whole stack, including the framework.
Tool: PHPUnit + Docker with all infrastructure elements needed.

We will implement functional tests for all endpoints.

Acceptance tests

Acceptance tests need context to be run and understood. They may consist on testing the same endpoint multiple times,
with different use cases. Acceptance tests check the behavior of the application, without regarding infrastructure
details. For example, we will not check the database if the tests does not say so.

These kinds of test should be able to be written using Gherkin syntax.

The difference between functional and acceptance tests is blurry, so the developer will have to decide each time the
place where the test belongs.

Who needs to understand them: product owner.
Apply to:
- Application and Domain layers, if we want to test only business rules.
- The whole stack, including the framework, if we want to test the system as a whole.
Possible tools:

End-to-end tests

End-to-end tests checks user behavior, or, as we greed on, an interface. In the case the project is an API, this kind of tests will not apply.

Who needs to understand them: user.
Apply to:
- The whole stack, including the framework and even the web server.
Tools:
- Cypress
- Selenium
- Panther

Benchmark tests

Benchmark tests check that a concrete functionality gives a response within an accepted threshold. For example, we
may want that a core endpoint call or function gives a response in less than 0.5 seconds.

Who needs to understand them: developer.
Apply to:
- Infrastructure/Application/Domain: if we want to test how performant the code is.
- The whole stack, including the framework and even the web server: if we want to test the whole infrastructure.
Tools:
- PHPBench (for code)
- JMeter (for infrastructure)

Summary

As said, these are the kind of tests that we agreed on, not pretending to be an exhaustive list nor an academic research. Thus, it is open to future modifications or insertions, depending on what we learn.

And you? Which kind of tests do you implement in your project? Do your testing strategies agree with ours?

Money pattern in PHP

Rubén Rubio — Mon, 28 Jun 2021 09:20:04 +0000

Introduction

As a business requirement, we needed to implement a new payment method. All our payments methods are in legacy code, so we took the opportunity to implement a way to manage amounts, trying to avoid problems we found in the past.

Besides an amount, a money always has a currency. When we say a product costs 7.99, what are we referring to? €7.99? $7.99? ¥7.99? The price of a product can vary a lot if we do not take into account the different currencies.

For small businesses that only operate in a country or a region with a single currency, it may have sense to always assume the use of this currency everywhere. If the application does not expand to a region with a different currency, this assumption will be safe. But we can never be a hundred percent sure that this will not happen in the future.

To solve this problem, we can use a pattern know as "Money pattern", that consists on having a Value Object that has two attributes: the amount and the currency.

Precision

Representing numbers in software is always a complex task. As numbers usually have decimal parts, our first intuition to represent an amount is to use floating point numbers. But floating point numbers have a limited amount of decimals they are able to represent, as the machine's memory is limited. For example, the number 2/3 is 0.666... At some point, the system will round the value, for example, to 0.666667. Therefore, there will always be some loss of precision when using floating point numbers.

We can try to solve this by rounding the numbers at some point during the execution.

PHP Functions

In PHP, we have multiple functions that allow to round numbers. These are some of them:

`floor`

floor($amount);

Returns the next lowest integer value (as float) by rounding down value if necessary

`ceil`

ceil($value);

Returns the next highest integer value by rounding up the value if necessary.

`round`

round($amount, $precision, $mode);

Returns the rounded value of val to specified precision (number of digits after the decimal point). Precision can also be negative or zero (default).

`number_format`

number_format($amount, $decimals);

Formats a number with grouped thousands and optionally decimal digits.

Example

Let's see a real example to see how this works.

I worked in an e-commerce project that had a back-office where shop administrators could set the base price (without taxes) for their products. The VAT amount (of 21%) and the final price were automatically calculated from that base price.

This concrete administrator wanted to have two different products, with a final price (including taxes) of €5.50 and €5.30. We had a user that bought 5 units of each product. So, ideally, the total price for that order would be €54.00:

Base price (€)	VAT 21% (€)	Total (€)
4.55	0.95	5.50
4.55	0.95	5.50
4.55	0.95	5.50
4.55	0.95	5.50
4.55	0.95	5.50
4.38	0.92	5.30
4.38	0.92	5.30
4.38	0.92	5.30
4.38	0.92	5.30
4.38	0.92	5.30
44.63	9.37	54.00

However, things were not that nice. The back-office form allowed the administrator to introduce three decimal point numbers in order to (unsuccessfully) avoid rounding problems. The values were rounded before saving. Therefore, the value that was sent to the bank to charge the client for was as follows:

Base price (€)	VAT 21% (€)	Total (€)
4.545	0.9545	5.4995
4.545	0.9545	5.4995
4.545	0.9545	5.4995
4.545	0.9545	5.4995
4.545	0.9545	5.4995
4.380	0.9198	5.2998
4.380	0.9198	5.2998
4.380	0.9198	5.2998
4.380	0.9198	5.2998
4.380	0.9198	5.2998
44.625	9.371	53.996

As you see, the client paid 1 cent less than she should have. It is not much, but in a store with thousands of transactions there would be a sensible loss. Fortunately (or not), it was not the case.

But things did not end there. Instead of doing a single calculation for the whole order and save the values in the database, the application was only saving the basic values (the unitary amount and the quantity for each line) and performing the calculations in each place they were needed. The problem here is that the calculations were done differently in each place, i.e., the application was rounding values earlier or later in the total calculation. So, even if the amount the client paid was €53.996, in the order confirmation email the application sent him, the amounts were calculated as following:

Base price (€)	VAT 21% (€)	Total (€)
4.550	0.9555	5.5055
4.550	0.9555	5.5055
4.550	0.9555	5.5055
4.550	0.9555	5.5055
4.550	0.9555	5.5055
4.380	0.9198	5.2998
4.380	0.9198	5.2998
4.380	0.9198	5.2998
4.380	0.9198	5.2998
4.380	0.9198	5.2998
44.65	9.38	54.03

So, the administrator expected to charge a total amount of €54.00, the client paid €53.996 whilst the total amount in the confirmation email showed €54.03.

Therefore, rounding values using PHP functions is not a suitable way of working with amounts (neither for numbers).

Libraries

Fortunately, in PHP there are open source libraries that solve these two problems, namely, working with potentially infinite numbers (up to a limit) and having an amount with a currency.

The ones we checked in depth were the following:

We finally chose brick/money because it uses underlying the package brick/math, that allows more robust calculations. There is a comparison between the two libraries, but you should be safe using either one of them, specially after moneyphp/money got a big rewrite one month ago (on May 2021).

Implementation

We implemented an Amount Value Object:

<?php

declare(strict_types = 1);

namespace SharedKernel\Domain\ValueObject;

use Brick\Math\BigNumber;
use Brick\Math\Exception\NumberFormatException;
use Brick\Math\RoundingMode;
use Brick\Money\Exception\MoneyMismatchException;
use Brick\Money\Exception\UnknownCurrencyException;
use Brick\Money\Money;
use SharedKernel\Domain\Exception\InvalidAmountException;

final class Amount
{
    private const ROUNDING_MODE = RoundingMode::HALF_UP;

    private int $amount;
    private string $currency;

    private function __construct(int $amount, string $currency)
    {
        $this->amount = $amount;
        $this->currency = $currency;
    }

    /**
     * @param BigNumber|float|int|string $amount
     * @param string                     $currency
     *
     * @return Amount
     *
     * @throws InvalidAmountException
     */
    public static function of($amount, string $currency): self
    {
        $amountAsMoney = self::parseAndValidateOrFail($amount, $currency);

        return new self(
            $amountAsMoney->getMinorAmount()->toInt(),
            $amountAsMoney->getCurrency()->getCurrencyCode(),
        );
    }

    public function equalsTo(self $secondAmount): bool
    {
        try {
            return $this->amount()->isEqualTo($secondAmount->amount());
        } catch (MoneyMismatchException $e) {
            return false;
        }
    }

    public function amount(): Money
    {
        return Money::ofMinor($this->amount, $this->currency, null, self::ROUNDING_MODE);
    }

    /**
     * @throws InvalidAmountException
     */
    private static function parseAndValidateOrFail($amount, string $currency): Money
    {
        try {
            return Money::of($amount, strtoupper($currency), null, self::ROUNDING_MODE);
        } catch (UnknownCurrencyException $e) {
            throw InvalidAmountException::withInvalidCurrency($currency);
        } catch (NumberFormatException $e) {
            throw InvalidAmountException::withInvalidAmountFormat($amount);
        }
    }
}

The amount has two scalar attributes:

amount: stored as int with the minor unit (cents). For example, €105.35 will be stored as 10535.
currency: the currency code defined by the ISO 4127 standard.

A Value Object, by definition, can not be constructed invalid. In our case, we take advantage of the brick/money library to validate any input value, and then we represent them with both amount and currency attributes. We can generate a Brick\Money\Money value from them.

As moneys have an official scale for the currency defined by the ISO 4127 standard. Therefore, the idea is to persist
both components as scalar values, because we can then generate a Brick\Money\Money value from them.
We can also encapsulate any required operations in SharedKernel\Domain\ValueObject\Amount.

Persistence

As moneys have an official scale for the currency defined by the ISO 4127 standard, the idea is to persist
both components, amount and currency as scalar values, and then reconstruct the Amount ValueObject from them.

There are different options to persist the amount depending on the persistence layer we use. We have a concrete case where we use Postgres with Doctrine ORM as an abstraction layer, so we chose to persist both attributes as different columns using an embeddable:

<?xml version="1.0" encoding="UTF-8" ?>
<doctrine-mapping xmlns="http://doctrine-project.org/schemas/orm/doctrine-mapping"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                  xsi:schemaLocation="http://doctrine-project.org/schemas/orm/doctrine-mapping
        http://doctrine-project.org/schemas/orm/doctrine-mapping.xsd">

    <embeddable name="SharedKernel\Domain\ValueObject\Amount">
        <field name="amount" type="integer" column="amount" />
        <field name="currency" type="string" column="currency" length="3" />
    </embeddable>
</doctrine-mapping>

This way, we can map the Amount Value Object to our entities easily. Another option would have been to create a custom type to persist both values as JSON in a single column. That would be the chosen option if we used a No-SQL storage.

Summary

On one side, we saw that managing numbers in computing is complex and prone to errors. We also saw that a money always has to have a currency, and that the money pattern precisely allows to do that.

On the other side, we saw that it is better to not reinvent the wheel and implement a custom solution, but to use libraries that are specifically designed for the task.

Finally, we saw an implementation of the money pattern in PHP as a Value Object, with a possible persistence strategy using Doctrine.