Enhanced your functional tests with Symfony (API context)

My team and I have a lot of micro services to develop and maintain, and, since the key of success for this kind of architecture is performance, it is our main concern while coding.

Unfortunalty, Symfony does not provide a simple way for monitoring performances metrics other than the Profiler.

TL;DR:

We wanted to be able to perform automatic checks on our performances:

• Number of database queries
• Execution time (and not only the response time, which include too much variables)
• Memory used

How to be fast ?

That's not the subject of this post, but I'll do a quick reminder about few requirements for a fast API, in a Symfony context:

• Have as less database queries as possible
• Don't have too much injected services
• Use as less as memory as possible
• Disable/remove all unused extra Symfony packages
• If possible, don't use Listeners
• ...

But, how'd you check that ?

Symfony provides a very handfull tool to check some of the previous bullet points.
It allows you to:

• Know the number of database queries
• Know how much memory your app is using
• Have information about performances

Amazing, right ?

When I'm coding or doing a code review, it's one of the first thing I check.
It's easy when you have a route to get all rows of a table, with a OneToMany relation to have an impressive number of queries, and that should not go to production !

What about functional tests ?

Well, your code is OK on your machine, you have the right number of queries, you know your route don't consume too much memory and that it's fast enough.

But I'm a paranoid developer, I don't trust code, not even mine. That's why I write unit tests, functional tests and integration tests.
And none of this tests allow me to assert that my route has less than 5 database queries, and that the execution time (which if different than response time) is fast enough.

I looked over, and I didn't find any easy way to do so. BlackFire.io do it, but it's expensive and a little bit too much for me.
But, hey, I'm a developer, so let's do it myself, that'll be fun !

First, where do I want to have access to this data ?

During unit tests ?

Nah, I don't want to do database calls during thoses tests, and, I prefer not have to boot the kernel

During integration tests ?

That might do the trick, but for now we don't go through CI before commiting, so that'll be triggered only before being push on production

During functional tests ?

Yes ! That'd be perfect.

We do that using Postman (and its CLI tool "newman").
I love this tool because it provide a way to test your work in progress and, when you're done, you just write tests in Javascript.

We are used to run the postman tests before commiting, during the code review (it's part of it), and, of course, before any push on production.

Accessing the data

Ok, I know when I want to have access to the metrics, I still have to figure how.
Postman cannot access the profiler, since it'd require a secondary HTTP call, and to crawl the HTTP response. So we had to figure out another way.

Why not pushing those metrics in the response header ?

• Postman can read them and perform some tests on it
• We can access them with ease, even from Chrome dev tool
• It's pretty sexy, let's admit it !

We decided that we want to have:

• The name of the controller and method called by the router
• Have the number of database query
• The memory used
• The execution time

Execution-Controller → App\Controller\MyController -> myMethod
Execution-Doctrine-Queries → 8
Execution-Max-Memory → 4194304 bytes
Execution-Time → 206 ms

How it works ?

Well, it's really simple.

• A StopWatch is started when any Controller is triggered using the onKernelController event, then stoped on KernelResponse. This StopWatch give us the Execution-Time AND the Max-Memory
• We use the DoctrineDataCollector (as the Profiler does) to count the number of queries

Add it to the headers, and that's it !

Write some tests in Postman or whatever tool you are using, and you can assert that your route is using less than 2mb of memory and require less than 5 database queries !

laboutiqueofficielle / MonitHeaderBundle

Add monitoring data in the response's header

MonitHeaderBundle

This Symfony 3 Bundle add some extra informations to the response's header

Installation

Simple add this line to AppKernel.php

 public function registerBundles()
    {
        $bundles = array_merge($this->servicesBundles($this->getEnvironment(), $this), [
            [...]
            new Lbo\Bundle\MonitHeaderBundle\MonitHeaderBundle(),

Usage

Then, inspect your headers with, for example, Postman

Execution-Controller →App\Controller\MyController -> myMethod
Execution-Doctrine-Queries →8
Execution-Max-Memory →4194304 bytes
Execution-Time →206 ms

View on GitHub

Comments

[Sergey Telpuk]

Let me suggest a useful solution to microservices. Please look at spiral/roadrunner. It can give you high performance.

[Valentin Silvestre]

Don't forget about using APP_ENV = prod, because log and db profiling can just burn your memory :)

[Xavier Dubois 🇫🇷]

Well, if you do so before running your functional tests, you won't be able to use the the MonitHeaderBundle since it's based on the same components than the profiler ;)

[Valentin Silvestre]

That will require extra work but profiling performance on dev is nearly useless :/