DEV Community: Charles

Approval testing: 3 technics to deal with randomness

Charles — Thu, 19 Jan 2023 14:33:26 +0000

This article was previously posted on my blog.

When dealing with legacy code, randomness is not something you enjoy meeting.

Creating a test harness on top of a system with random results can be pretty challenging.

Obviously, you’d like your tests to be deterministic and to always give you the same result, provided you didn’t change anything in the system.

Let’s see three ideas that can help you deal with the randomness in the system. The first two technics are based on the idea of removing the random part of the system during tests.

Block the random generator

The first solution I wanted to offer in this article is to ensure that the random generator always returns the same results when called. The answer here is to seed the generator with a value you control, to make the generator work in a deterministic manner.

For instance, in PHP, you can call the mt_rand function. Later, any call to the rand function will return the same random numbers in the same order.

This is a fast way to turn the non-deterministic system into something that always gives the same results.

Unfortunately, this solution, while being one of the quickest ways to get the job done, only works in some situations. First, not all languages allow seeding the random number generator. If you can’t, you can’t and need another solution.

Another caveat of this solution is that if you modify the structure of your code and the order of calls to a random method is changed; you’ll get results in the same order but not in the same logical space. For instance, let’s take two calls to rand and assign the result of the first call to variable A and the second call to variable B. After seeding the random number generator, A and B are always set to the same value, say 34 and 1879. During a refactoring, if we decide to invert the order assignation, we have a problem. The code now assigns values to B before A; A is now 1879, and B is 39. You can play with this idea here.

What can we do when seeding the random number generator is impossible, or we might want to change the order of the calls at some point?

Control the randomness

The other alternative is to take back control of the values generation. As often, adding a layer of abstraction is one of the solutions available in our toolbox. Here we want to break the direct dependency to the random value generator and swap it with a dependency we can manipulate.

In more concrete terms, instead of directly calling a function from the language, we encapsulate it, create a stub we can manipulate, and find a way to substitute the original implementation with our double.

The substitution technics available greatly vary depending on the programming language you’re using and the code’s state. If you’re lucky enough to be able to inject dependencies at construction, this is an easy task. If you’re not, some patterns from Working Effectively With Legacy Code by Michael Feathers can prove helpful.

This solution is available in every language. The only requirement is to feel comfortable messing around with code while not having tests to be able to take control back.

Once you have control, you can decide the values generated.

That solution allows us to solve the ordering issue mentioned before. If you invert the order of calls in the code, you can change the configuration of your stub to change the order and keep the tests passing.

Modifying the stub after tests are created needs to be done carefully. If you want greater confidence and want to avoid touching the tests, you have another alternative. Instead of making one abstraction for every call to the random generator, you can create one for each call. Taking back the example from the first part, you can introduce an abstraction for generating A and an abstraction for generating B. Having two implementations gives you more granular control, and you can create two stubs. When you invert the call order, the code still calls the right abstractions and the same stubs and obtains the same final result.

Touching an existing code base to introduce indirections to control randomly generated value is not always straightforward nor necessary. Sometimes we don’t care about the generated value, and we can be totally fine with keeping the randomness as long as we manage to keep our tests deterministic. And this is the point of the third idea.

Hide the randomness

Sometimes the random value is not really important for the result. If we don’t care about it, we can remove it from what is asserted in the test.

The trick here is that you don’t need to make assertions based on the direct outputs of the system under test. Instead, you can get them and transform them to keep only the information you need to be confident enough that you’re not breaking anything.

When doing approval testing, it is common to use a printer, a function or a class that will transform the outputs to make them look easy to understand. If a piece of information is not relevant, the printer’s role is to remove it and create a clean output. You can then use that clean output in the assertion.

What could be simpler than removing what you don’t care about?

Sometimes you don’t care about the actual value, but you want to ensure that a value is here. Instead of removing the random part, you can replace it with something else. Say you don’t care about the value of a UUID but want to be sure that it is displayed. Using a regular expression, you can replace all UUIDs with something else, like a UUID string.

Another common use case is to detect that a value is repeated in several places in the output.

Let’s continue with the previous example. You still don’t care about the value of the UUID, but you want to assert that the UUID used in a link to access a product is the product’s UUID, not some other one.

Here the printer has to be more clever.

When a UUID is detected using a regex, the scrubber generates a new string, usually made from a base string and an increment (UUID_1). Then it replaces all the subsequent occurrences of that UUID with the generated string, increments the counter and repeats for each UUID found in the output. That way, even if you can’t predetermine the result of the system, you can turn it into something deterministic that you can use as your golden master.

The good thing about this solution is that it doesn’t require touching the legacy system. You don’t need to worry about breaking something while creating the tests. This is why it is a very convenient and relatively rapid method to build confidence. Also, you might be very lucky and work with a language where approval testing tools come with scrubbers doing part of the cleaning work for you.

Discovering a system using randomness and needing to create a test harness doesn’t have to be scary or take an awful lot of time. With the three ideas exposed in this article, you should be able to attack that piece of legacy code you avoided refactoring for a while.

Comment tester une méthode privée ?

Charles — Wed, 21 Sep 2022 11:16:10 +0000

Cet article a été posté précédemment sur mon blog.

Dans cet article quand j’utilise le terme "méthode privée" cela correspond aux méthodes qui ne sont pas accessibles depuis l’interface publique d’une classe. En PHP, par exemple, il peut s’agir de méthodes private ou protected.

On ne la teste pas. Ça, c’est la réponse courte. La réponse longue demande un peu plus d’effort.

Commençons par le commencement.

La question originale est trompeuse. En réalité on ne cherche pas à tester une méthode, quelle soit private, protected ou public, mais un comportement. On veut savoir ce que fait notre logiciel, pas comment il le fait, et donc la manière dont on implémente ce comportement nous importe assez peu.

Évidemment, on va quand même retrouver une méthode publique comme point d’entrée du comportement, mais la méthode privée, elle, est un détail d’implémentation. Le comportement qui nous intéresse pourrait tout aussi bien être servi par une méthode publique qui contient l’ensemble du code nécessaire, ou une méthode publique qui délègue une partie du travail à une ou plusieurs méthodes privées, ou même encore à une ou plusieurs autres méthodes publiques d’autres classes. En somme, l’existence ou non d’une méthode privée ne nous intéresse pas.

Pour tester le comportement, on va donc se placer à l’endroit où l’on peut le déclencher et vérifier son bon déroulement, c’est-à-dire au niveau de la méthode publique. Au passage cela va "tester la méthode privée", ou plus précisément "exercer le code contenu dans la méthode privée". Encore une fois, on ne veut pas tester une méthode privée mais un comportement qui s’appuie sur une méthode privée. On peut dire que l'on "teste" la méthode privée via une méthode publique.

Oui, mais cette méthode privée est complexe

Ceci étant dit, il se peut que le code présent dans la méthode privée soit assez complexe pour que l’on ait envie de pouvoir exercer ce bout de code sans avoir à passer par la méthode publique. On veut ainsi éviter de devoir faire une mise en place de test complexe, d’avoir à zigzaguer au milieu du reste du code pour pouvoir se placer dans un cas particulier du comportement qui nous intéresse.

Une solution couramment proposée est d’essayer de contourner le problème en rendant la méthode privée accessible. Ce qui peut se faire en utilisant le système de réflexion du langage. L’avantage de cette solution est que l’on n’a pas à toucher au code. En revanche on ajoute de la complexité à la mise en place de notre test et l’on augmente le couplage entre les tests et le code. Cela peut alors nuire à un potentiel futur refactoring, puisque les tests ont alors une connaissance assez fine de l’implémentation.

Une autre solution, bien meilleure selon moi, est d’écouter les tests. Comme très souvent, quand quelque chose est compliqué à tester ce ne sont pas les tests le problème mais la conception du code.
En nous montrant qu’un bout du comportement est complexe, si complexe que nous avons envie de pouvoir nous assurer de son bon fonctionnement séparément du reste les tests nous indiquent qu’il nous manque un concept, qu’il manque un collaborateur.

Il faut alors prendre le temps de trouver quel est ce concept, de le définir, de le nommer. On peut ensuite introduire ce nouveau concept dans notre code à l’aide d’une nouvelle classe et y transférer la méthode privée, que l’on rend publique au passage.

Cela nécessite, certes, de modifier le code, mais permet de faire naitre un vocabulaire plus riche pour discuter du problème que l’on cherche à résoudre. On est capable de documenter cette partie du comportement par elle-même au travers des tests.

On n’a pas non plus besoin d’utiliser d’astuces qui complexifient les tests, un simple appel à la nouvelle méthode publique suffit.

On a réussi à transformer notre problème. Puisqu’il n’y a plus de méthode privée on n’a plus besoin d’essayer de tester de méthode privée.

Voilà pourquoi la réponse longue à la question "Comment tester une méthode privée ?" est aussi "On ne la teste pas".

Envie d'en apprendre davantage sur les tests ?

J’ai créé une formation vidéo qui aide les développeuses et développeurs à améliorer leurs tests automatisés.

Dans cette formation je partage les idées et techniques qui permettent de rendre des tests lents, qui cassent à chaque modification du code et sont incompréhensibles en des tests avec lesquels on a plaisir à travailler.

Generating dates and UUIDs easily in PhpStorm

Charles — Fri, 13 Aug 2021 10:22:53 +0000

This article was previously posted on my blog.

Legacy codebases tend to lack tests. One way to introduce some safety net before messing around with the code is to add characterization tests.
In this kind of test, we feed some input to the system under test and look at the outputs, ensuring that they stay the same test run after test run. It's far easier to fix inputs once and for all.

When creating this type of test, we want to avoid moving or random values. Two of the most common are dates and UUIDs. While it's possible to generate UUIDs using external tools, there is room for improvement if you want to do everything directly in PhpStorm. Even if typing a date is not as painful, we can gain some time here too.

PhpStorm, and probably all IntelliJ IDEs, allow creating Live templates. Live templates are templates that we can expand by typing an abbreviation and triggering the autocomplete. What's even more remarkable is that they can evaluate expressions when the abbreviation is expanded. This is what we'll use.

Generating the current date

For generating the current date, we can use the date expression provided by PhpStorm and bind it to a variable.

In PhpStorm settings, go to Editor > Live Templates
Expand the PHP section
Click on the Add button, + on the right side, and select Live template
In the abbreviation field, enter $date$ . This is what we'll type to expand the template.
In the template text field, enter '$date$'$END$.
Click Edit variables
In the line for the variable named date, add the expression date("yyyy-MM-dd'T'HH:mm:ss.SSSz") and select Skip if defined
Click Define below the ⚠️ No applicable contexts
Select PHP > Expression
Save and close the settings with the Ok button.

Generating a random UUID

Generating a UUID is slightly different because PhpStorm doesn't provide a function to create UUIDs directly, but it, fortunately, allows to run Groovy scripts.

In PhpStorm settings, go to Editor > Live Templates
Expand the PHP section
Click on the Add button, + on the right side, and select Live template
In the abbreviation field, enter $uuid$ . This is what we'll type to expand the template.
In the template text field, enter '$uuid$'$END$.
Click Edit variables
In the line for the variable named uuid, add the expression groovyScript("UUID.randomUUID().toString()") and select Skip if defined
Click Define below the ⚠️ No applicable contexts
Select PHP > Expression
Save and close the settings with the Ok button.

And voila, we now can quickly insert the current date or a UUID in our code.

Thanks to PhpStorm allowing us to execute Groovy code, we can probably think of other ideas and save more time. For instance, it would probably make sense to generate a random date instead of the current one, as we could be in a particular case if the code has conditions based on the current date.

An alias to learn aliases

Charles — Sun, 20 Jun 2021 13:46:56 +0000

This article was previously posted on my blog.

I’m trying to save some time by reducing the number of keystrokes I need to do something.

I’m pretty efficient with my IDE’s shortcuts, but I have vast room for improvement in my console.

The obvious solution there is relying on aliases.

I’ve been using the git ssh plugin for several years, but I only use a handful of the provided aliases, mostly because I never took the time to memorize past the few main ones.

Now that I’m trying to save some typing, I’m learning the other ones. I have an issue, though: I spend a lot of time looking at the documentation to know which alias shortens the command I want to enter. In that case, it means I switch to my browser, get the right tab, or go to the website, use the search to find the suitable alias by typing part of the command. I don’t save a lot of keystrokes, and I’m surely not gaining time.

Please don’t take me wrong; time spend learning or practicing to improve in the long run is time well spent. Nevertheless, I decided to find a way to enhance my learning process.

The best way to reduce the amount of time needed to find the proper alias for a command is to stay in the console and avoid looking at the documentation.

After a quick search, I discovered that the alias command lists all aliases set up. That’s a start.

Combining alias and grep makes it efficient to find the correct alias for the job.

That still requires some typing, if your shell doesn’t have some sort if autocomplete at least.

As I’m in a quest for laziness and poking around with aliases, I decided to add a new one, fa, for “find alias” defined as alias fa='alias | grep'.

An example: searching for all aliases about stash

Just two letters, no more windows switching, and a lot of time saved.

Do you have some tips about productivity as well?

Announcing Burritalks.io, a curated list of talks for product people

Charles — Mon, 08 Feb 2021 17:00:31 +0000

I read many books and blog posts and watch a lot of talks about software product creation. People I've worked with know that I always have resources to share on many subjects related to software product creation.

I started keeping a list of exciting talks a few years ago. Speeches that made me think, gave me new perspectives or ideas, reshaped the way I think about building software, and helped me grow as a developer.

After a while, I stopped adding items to the list. Even if it was public, no one was using it, not even me. It wasn't easy to find nor looking good. I decided to trust my memory and others medium - Twitter or Pocket - as good places to keep track of these talks. Time proved me wrong.

I hope that creating and curating a website will help me commit to keeping track of them.

The content is primarily oriented for developers. A lot of talks are about system architecture, testing, agile transformation, and choosing the right tool for the job.

So, here it is: Burritalks.io !

I hope you'll get as many insights as I got from watching some of this content. Let me know what you think!

Edit: fixed link 😅

Faster environment with xDebug and Docker

Charles — Tue, 19 Jan 2021 18:25:37 +0000

I have to admit that I've always been lazy when it came to set up xDebug as it felt tedious the few times I've to do it. It made me an almost perpetual member of the var_dump debug team.

Nevertheless, the last time I had to set it up was more straightforward, thanks to PhpStorm, and I don't think the difficulty to set up is a valid excuse anymore.

I then had a debugger…

And I had a speed issue. My tests were running slower than before, and every request took way too much time. Using Docker on OSX didn't help at all.

I found a few tips that helped me go back to the same state as before.

So …

Tips 1: Do not start xDebug

Ahah… Yes, you can thank me for this one. The best way to avoid xDebug slowing everything down is by not having xDebug running. So, disable it by commenting the path to the extension in the .ini file loading it. Another idea is that if you have an xdebug.ini file in the PHP configuration directory, rename it to something else, xdebug.ini.back, for instance.

This being said, how can we manage to start xDebug when needed?

Tips 2: Use the on-demand PhpStorm mode when debugging tests

As explained in the PhpStorm documentation, an "on-demand" mode is available. It will enable xDebug only when debugging tests, which will allow running your test at full speed most of the time.

I'll let you dig in the documentation as it will be better explained than if I do it myself.

Tips 3: Use the on-demand mode without PhpStorm

PhpStorm is not doing magic for the "on-demand" mode but taking advantage of the -d PHP CLI option. The -d allows defining an entry in the PHP configuration. When starting the "on-demand" mode, PhpStorm adds -d zend_extension=xdebug.so to the PHP CLI options, which enables the extension. You can do the same thing if you're running your tests without PhpStorm.

As an example here is how you can run PhpUnit with xDebug activated:

php -d zend_extension=xdebug.so ./vendor/bin/phpunit

You'll probably need to pass more options if you want to connect xDebug to a debugger client, so this not as convenient as running it through PhpStorm as it sets everything needed to work out the box. As an example, here are all the option passed by PhpStorm when I start tests in debug mode :

-d zend_extension=xdebug.so -d xdebug.remote_enable=1 \
-d xdebug.remote_mode=req -d xdebug.remote_port=9100 \
-d xdebug.remote_host=host.docker.internal

This is still a good option if you need xDebug without a client listening. I guess the primary use case would be for coverage reports.

Tips 4: Enable xDebug with an environment variable in your docker container

Going in the container and modifying files every time you need to enable or disable xDebug is cumbersome. I've tested another option for the last few weeks, and it works very well.

I've modified my PHP Dockerfile to add an entry point script, which is run when the container starts and can access environment variables.

The entry point script looks for an environment variable and decides to rename, or not, the xdebug.ini file in the PHP configuration directory.

As an example, here is the entrypoint.sh file I'm using in a php5.6 container. You'll probably need to adapt it to suit your container.

#!/bin/bash
set -e

if [[ ! -z "$DISABLE_XDEBUG" && "$DISABLE_XDEBUG" = true && -f "/etc/php/5.6/mods-available/xdebug.ini" ]]; then

mv /etc/php/5.6/mods-available/xdebug.ini /etc/php/5.6/mods-available/xdebug.ini.back

cat >&2 <<EOF
⚠️  xDebug is disabled
EOF

elif [[ -z "$DISABLE_XDEBUG" || "$DISABLE_XDEBUG" = false && -f "/etc/php/5.6/mods-available/xdebug.ini.back" ]]; then
mv /etc/php/5.6/mods-available/xdebug.ini.back /etc/php/5.6/mods-available/xdebug.ini
fi

exec "$@"
And the Dockefile is edited to include the entry point:

COPY ./entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
CMD ["php-fpm5.6"]

The container can be started either with docker-compose or docker run and xDebug activated by setting the environment variable DISABLE_XDEBUG to false and disabled by setting it to true. When the variable is missing, xDebug will stay activated.

We now have a convenient way to enable xDebug only when needed and save some precious time not waiting in front of loading pages. The only thing required is to change the value of the variable and restart the container.

Extra tip

xDebug settings can be overridden using the XDEBUG_CONFIG environment variable, which means there is no need to update the xDebug config file every time you have to change a setting or if you and your teammates need different settings.

The best example is probably the remote_host. According to how you run docker, this setting could be a changing IP address or host.docker.internal if you're using Docker for Mac.

In order to set the remote host to host.docker.internal the XDEBUG_CONFIG should be set remote_host=host.docker.internal.

I hope all these tips will help you improve the speed of your development environment.

This post was originally posted on my blog

Picture by Paolo Candelo