Consistent validation with API Platform 3

TL;DR

API Platform uses two Symfony components - Serializer and Validator - when dealing with requests. By default, these components produce different error responses which can lead to inconsistency in an API. Configuring denormalization errors' collection allows to transform type check exceptions into proper violations list.

Failing tests

API Platform is a great tool for rapid API development, but it has a lot of not-so-well-documented features which can sometimes lead to confusion. Playing around with a new project of mine I've stumbled into one: tests were failing for my validation assertions of endpoints' responses!

All code blocks are modified examples from this repo. Feel free to explore it for more details.

Before we continue let's see some code we're talking about.

// src/Entity/Vacancy.php

<?php

declare(strict_types=1);

namespace App\Entity;

use ApiPlatform\Metadata\ApiResource;
use Symfony\Component\Serializer\Attribute as Serializer;
use Symfony\Component\Validator\Constraints as Assert;

#[ApiResource(
    normalizationContext: ['groups' => ['read']],
    denormalizationContext: ['groups' => ['create']],
)]
class Vacancy
{
    #[Serializer\Groups(['read'])]
    public ?int $id = null;

    #[Serializer\Groups(['read', 'create'])]
    #[Assert\NotBlank, Assert\Length(max: 255)]
    public ?string $title = null;

    #[Serializer\Groups(['read', 'create'])]
    #[Assert\NotBlank, Assert\Positive, Assert\Type('integer')]
    public ?int $minBudget = null;
}

What we have here is a perfectly boring API resource (which is also a Doctrine entity in the source code, but here it's irrelevant). It has serialization groups and some basic validation. Any validation errors should trigger a 422 UnprocessableEntity API response, so we'd expect that providing invalid values (say {"minBudget": -111.34}) will do just that. Let's test it!

// tests/Functional/Api/Vacancy/CreateVacancyItemTest.php

<?php

declare(strict_types=1);

namespace App\Tests\Functional\Api\Vacancy;

use ApiPlatform\Symfony\Bundle\Test\ApiTestCase;

final class CreateVacancyItemTest extends ApiTestCase
{
    public function testValidateCreateVacancyRequest(): void
    {
        self::createClient()->request('POST', '/vacancies', ['json' => [
            'title' => 'Perfectly valid vacancy title',
            'minBudget' => -111.34, // Absolutely invalid value
        ]]);

        $this->assertResponseIsUnprocessable();
    }
}

So far everything looks good. Negative float should trigger constraint violation with 422 response status and some message like 'This value should be int'. I'm not going to verify exact response message here, at this point it's quite enough to know that my frontend app can rely on the API returning 422.

Running test and... What? Why?

PHPUnit 9.6.19 by Sebastian Bergmann and contributors.

Testing App\Tests\Functional\Api\Vacancy\CreateVacancyItemTest
F                                                                   1 / 1 (100%)

Time: 00:00.878, Memory: 48.50 MB

There was 1 failure:

1) App\Tests\Functional\Api\Vacancy\CreateVacancyItemTest::testValidateCreateVacancyRequest
Failed asserting that the Response is unprocessable.
HTTP/1.1 400 Bad Request

It took the whole day for me to sort this out.

Rooting the problem

For now, we have two Symfony components (Serializer and Validator) used in accordance with their respective documentation and the guides from API Platform itself. Everything is telling us that we should receive another error with another status and a list of meaningful violations!

Is it even important? An error is an error. Well, these responses have different structure. Symfony Validator produces a responses with an array of violations:

{
    "status": 422,
    "violations": [
        {
            "propertyPath": "minBudget",
            "message": "This value should be of type int."
        }
    ]
}

And in case of API Platform the response will also have Hydra metadata and JSON-LD context, specifying that we have a validation error.

The Symfony Serializer component, on the other hand, doesn't create an array of violations. I guess it's tolerable in some cases, but the main point is to use single validation error format instead of mixing them and that we're obviouly not getting right now.

Before digging deeper I'd like to extend my test to cover more cases. I'll use a data provider to run the same test with different input sequentially.

// tests/Functional/Api/Vacancy/CreateVacancyItemTest.php

<?php

declare(strict_types=1);

namespace App\Tests\Functional\Api\Vacancy;

use ApiPlatform\Symfony\Bundle\Test\ApiTestCase;

final class CreateVacancyItemTest extends ApiTestCase
{
    /**
     * @dataProvider requestBodyCases
     */
    public function testValidateCreateVacancyRequest(array $requestBody): void
    {
        self::createClient()->request('POST', '/vacancies', ['json' => $requestBody]);

        $this->assertResponseIsUnprocessable();
    }

    public static function requestBodyCases(): \Generator
    {
        yield 'invalid title' => [['title' => str_repeat('a', 256), 'minBudget' => 10]];
        yield 'invalid budget' => [['title' => 'Perfectly valid vacancy title', 'minBudget' => -111.34]];
    }
}

Well that's weird. Seems that the title field is validated as expected and the minBudget is not.

PHPUnit 9.6.19 by Sebastian Bergmann and contributors.

Testing App\Tests\Functional\Api\Vacancy\CreateVacancyItemTest
.F                                                                  2 / 2 (100%)

Time: 00:03.559, Memory: 74.50 MB

There was 1 failure:

1) App\Tests\Functional\Api\Vacancy\CreateVacancyItemTest::testValidateCreateVacancyRequest with data set "invalid budget" (array('valid', -111.34))
Failed asserting that the Response is unprocessable.
HTTP/1.1 400 Bad Request

Six hours (of blindly commenting out validation attributes, dd-ing around the code, etc.) later...

Exploring stack trace

To be honest I'm the kind of developer who thoroughly inspects the framework's error stack trace only when really desperate. Usually errors are on my side, so the first 1-2 entries are enough. Debugging is not really my thing too (especially in event based frameworks like Symfony where you could jump from listener to listener for half an hour before reaching something).

So when I'm telling that I started to carefully read the stack trace and manually inspecting vendor source code on lines specified there, that is desperation.

Fortunately (but not immediately) it was helpful. In the API Platform's decorators for Symfony Serializer I found the following lines:

try {
    $this->setValue($object, $attribute, $this->createAttributeValue($attribute, $value, $format, $context));
} catch (NotNormalizableValueException $exception) {
    // Only throw if collecting denormalization errors is disabled.
    if (!isset($context['not_normalizable_value_exceptions'])) {
        throw $exception;
    }
}

And this NotNormalizableValueException is the exception from the stack trace. After some googling and experimenting with global configuration I've discovered that this context can be set per resource or per operation directly in the ApiResource attribute.

// src/Entity/Vacancy.php

<?php

// ...

#[ApiResource(
    normalizationContext: ['groups' => ['read']],
    denormalizationContext: ['groups' => ['create']],

    // THIS IS IT
    collectDenormalizationErrors: true,
)]
class Vacancy
{
    // ...
}

Setting it resolved all issues with inconsistent validation error responses by skipping Serializer exceptions.

Final thoughts

It's worth explaining why fields validation behaved differently. As you may have noticed, the Serializer goes first: it transforms incoming JSON into ApiResource object (Vacancy in our case) and in the process performs a type check, utilizing PHP's is_{type} functions. Hence the value of title, invalid from the perspective of Validator component (we've tried to give it too long string), was a valid string which passes is_string($title) check. Integer type check, however, is more strict in this context and doesn't tolerate values wrapped in quotes, round floats, etc.

Any of the #[Assert\Something] rules will work only if the raw input was denormalized into an object. It is really a matter of separation of concerns.

Serializer creates valid object from raw data and will throw exceptions if it's impossible from data manipulation perspective.

Validator receives already created object and throws exceptions if object's values are invalid from application rules perspective.

I haven't found a way to enable the collectDenormalizationErrors globally. Some sources say that this can be done in api_platform.yaml configuration file with this exact parameter. It's obviously true, since setting this parameter in config file doesn't trigger any Symfony error like 'unknown configuration option', but it doesn't trigger the desired behavior for me either. If someone does know how it can be done, please do leave a comment.

Comments

[Alessandro Morelli]

Perhaps it can be set via the default serialiser context:

framework:
  serializer:
     default_context:
       collect_denormalization_errors: true

[Vsevolod Girenko]

Good idea, thank you, will definitely try it!

[Vsevolod Girenko]

Sadly, no, this didn't work. I'll keep looking for some other options. Thanks again for the suggestion!