The company I work for is a data-driven company. It is necessary for us to make the best decisions and aim for our core value: being customer-centric.
Naturally, when we iterate on a feature, we study the data to define how to improve it. And because we can’t improve what we don’t measure, we measure the results of our new version with the previous version.
To do this, we almost systematically perform AB tests that we couple with our events database.
AB tests are not for experimenting things, but to measure them.
Next part of this article is to start your first AB test, like changing the subject of an email when a new user registers.
The experiment we are doing will be called
welcome-email-experiment and it will have a
control version (the actual version) and a
participant (or variant) version (the new version we want to try).
When we will send this email, we’re going to require the VariantRetriever and ask it: for this experiment and this user identifier, tell me which variant it should have.
First step is of course to install the AB test package in your project:
composer require travaux-com/variantretriever
Because it’s not a Symfony bundle, you don’t need to declare it in your AppKernel file, but you still need (recommanded in fact) to declare it as a service.
We’re going to declare it as a service with the Symfony container factory system to have our “variant retriever” with all information about our running AB tests. (
services: App\FeatureFlag\VariantRetriever: factory: ['@Travaux\VariantRetriever\Factory\VariantRetrieverFactory', 'createVariantRetriever'] public: true arguments: - welcome-email-experiment: - control-email: 50 - participant-email: 50 - display-cta: - control-cta: 75 - participant-cta: 25
The service name is actually
App\FeatureFlag\VariantRetriever but I strongly encourage you to customize it.
Now you’re able to use the Symfony autowiring to retrieve the VariantRetrieverInterface and use it in your command handler.
And so on, how to use it ?
$affectedVariant = $variantRetriever ->getVariantForExperiment( new Experiment('my-ab-test'), (string) $user->getId() );
VariantRetrieverInterface actually came with the getVariantForExperiment method where you will have to define the experiment you want to test (by just filling the experiment name) and the “user identifier” who just need to be a string, so it can be a UUID v4 or an auto-incremented integer.
It will return you a
Variant value object that contains the name of the affected variant.
Now you can imagine select the right translation key to use depending on the affected variant for your email:
$experimentTranslationsKeys = [ 'control' => 'email.welcome.subject_control', 'variant' => 'email.welcome.subject_variant', ]; $userVariant = $variantRetriever ->getVariantForExperiment( new Experiment('welcome-email-experiment'), (string) $user->getId() ); $emailSubject = $experimentTranslationsKeys[(string) $userVariant];
This package will ensure you to always retrieve the same variant for your user, without any database or cache.
You’re now able to run your first AB in your Symfony project 🎉
Travaux\VariantRetriever\Retriever\VariantRetrieverInterface: class: Travaux\VariantRetriever\Retriever\VariantRetriever calls: - addExperiment: - !service class: Travaux\VariantRetriever\ValueObject\Experiment arguments: - 'welcome-email-experiment' - !service class: Travaux\VariantRetriever\ValueObject\Variant arguments: ['control', 100] - !service class: Travaux\VariantRetriever\ValueObject\Variant arguments: ['variant', 0]
Thanks to php crc32 function, we’re able to process a string that return everytime the same integer value. So on, even if it’s a big number, we’re able to reduce it under a range of 0 to 99 that match the rollout percentage given by one of our variant.
Travaux\VariantRetriever\Retriever\VariantRetriever class is not final to allow you to extend it. Like injecting a PsrLogger and/or the event dispatcher to keep a trace of which experiment have been affected to which user. You can also add you own method to retrieve a variant based on your User entity object like this:
public function getUserVariantForExperiment( Experiment $experiment, User $user );
Feel Free to contribute to this project or give feedback.