How does A/B testing work?

Mon, 19 Oct 2020 13:23:12 +0000

I wanted to have a much more implied answer to this question for the non-techy ones who are not much familiar with the topic but find it fascinating. This is my attempt to define A/B testing, as I know it. Feel free to correct me.

A/B test experiments are strictly based on a data-oriented approach. You use your already stored data to determine the areas lacking the mass appeal to convert the audience into customers. You apply the A/B test and receive the data about if the new version is bringing in some conversions or not.

What A/B testing does in all this is place triggers on the website, which activates the changes placed while building the experiment(with the help of codes and A/B testing tools and more), after a set action that could be a click on the page, while navigation to another page in the website, in form of pop up, etc.

That way the test loads on the screen of the random visitor (who fell in the variation’s bucket). A/B testing is a carrier as well as an agent between idea and conversion.

The basic ‘A/B testing’ works as a mechanism to enable testing of optimization efforts that to validate the ideas behind that change. To prevent the confusion it goes step-by-step.

Identifying, Ideation, and Hypothesis

To find the direction for your experiments, analyze your existing data to find the areas which could use some optimization. Within your analytics, you can find out where you have the most traffic and which pages convert the most by observing screen recordings, heatmaps, analytics, and hosting site data.

After checking the pages you finalized for experimentation, you’ll need to come up with ideas for the variations and what changes should be implemented. The term and process which is used for the idea generation is Conversion rate optimization.

Those ideas could use a hypothesis to convey the motive behind the change. That will help the developers understand your perspective while moving on with the creative part.

Experiment Building

Experiment building is done per the requirements of the presented idea to be mirrored, where the design files will be perfectly converted into HTML/CSS. Other necessary skills, such as JavaScript, coding, UI, and UX graphics, and more, are used to build a high-quality experiment. The developers ensure the experiments work well in all applicable devices and browsers.

Implementation and Analyzing

The test is implemented on the site per audience targeting. It’s only viewed by a specific set within your audience. Those who fall under the version’s bucket (the audience which fell in the variation’s bucket during traffic allocation) will be able to see it; others will see the original version only.

The performance of the tests has to be tracked until the experiment gains enough data for analysis. It takes some time to collect the sample size to evaluate the results.

The experiment records the results in the reports, those results are the KPIs like clicks, views, submissions, lead generation, sales, exits, and more. The winner between the original version and the Variation version (the one which includes the change) is determined on the basis of those results recorded in the reports.

If results are positive then the variation replaces the original and continues to increase the expected conversion rates.

The process may vary as per the experiment’s nature, type, and skills used. But overall it all depends on the data and how you make use of it to convert more users into customers.

via Brillmark

The Role of A/B Test Experiments in Growth Marketing

Tue, 29 Sep 2020 10:43:51 +0000

I just discovered this website. Interesting community. I am not a developer myself but somehow write about A/B testing. Would love to explore aspects of it. As of now, I am familiar with basic A/B testing and types. I wanted to try posting something. This isn't a typical 'technical' piece of the article but something I wrote about recently. I hope next time I could come up with something more useful.

The main post is this
Growth marketing is the term used to define the process of experimenting and improving a website to grow the customer base — all by just using data and metrics.

When we say “understand your target audience,” we mean “look into your analytics.” Check audience behavior; know the sources of your leads, traffic, and clicks; and the list goes on. Now, once you know the areas which can use improvement, run an A/B test experiment for those areas. Find out users’ reactions to those improvements and if they seem good enough. Apply it on the site. Then that area is certainly going to perform better than the previous version, as it is optimized perusers’ liking.

To explore more, there exist many kinds of experiments, such as A/B tests, multivariate tests, funnel testing, design experimentation, personalization, split-URL tests, and more. And to know what among these tests you need, you’ll have to know where you need it and why you need it.

Step 1. Identifying and Setting Up Goals
To find the direction for your experiments, analyze your existing data to find the areas which could use some optimization. Within your analytics, you can find out where you have the most traffic and which pages convert the most by observing screen recordings, heatmaps, analytics, and hosting site data.
For example, You could optimize the navigation bar to reduce bounce rates after observing an exit rate after users click on it.

This will help categorize your audience into different criteria, which further helps you track goals while experimenting. That way, you can identify the areas which could use some alterations to perform better. For example, the web pages with the highest bounce rates could benefit from experiments to make them more engaging.

Goals are necessary so you have a motive behind the whole process. It could be lowering bounce rates, reaching a minimum conversion rate, hitting KPIs, or whatever suits the experiment the best per the analysis.

Step 2. Ideation and Hypothesis
After checking the pages you finalized for experimentation, you’ll need to come up with ideas for the variations and what changes should be implemented. Conversion rate optimization is a great way to find ideas and additional elements specifically suited for your business and users. Your ideation and rough designs than could be mirrored in the next steps: experimentation.

The whole experimentation process is done to provide a better user experience, which eventually contributes to increasing the audience, conversions, and revenue. Those ideas could use a hypothesis to convey the motive behind the change. That will help the developers understand your perspective while moving on with the creative part.

Step 3. Experiment Building
The experiment ideas (along with hypotheses and goals) provide you with information about the targeting and A/B testing tools of your choice, but it may be necessary for A/B test developers to have access to set up the experiment. Experiment building is done per the requirements of the presented idea to be mirrored, where the design files will be perfectly converted into HTML/CSS. Other necessary skills, such as JavaScript, coding, UI and UX graphics, and more, are used to build a high-quality experiment. The developers ensure the experiments work well in all applicable devices and browsers.

Step 4. Quality Assurance
Once the development team is satisfied with their work, they send it for review to a quality assurance expert on A/B tests. The review is important to keep the experiment flicker, bug, and error free. If it doesn’t pass the review, the developer team reworks it and resolves the issues pointed out by the Q/A expert. It has to tick all the boxes of the Q/A checklist to ensure it will go live on the site without problems.

Step 5. Implementation and Analyzing
The test is implemented on the site per audience targeting. It’s only viewed by a specific set within your audience. Those who fall under the version’s bucket (the audience which fell in the variation’s bucket during traffic allocation) will be able to see it; others will see the original version only.

The performance of the tests has to be tracked until the experiment gains enough data for analysis. It takes some time to collect the sample size to evaluate the results.

The new experimental version has to complete all the designated goals in order to win and replace the original. If it fails to do so, more alterations could be made by using the existing data about the audience’s actions and response to it.

Step 6. Final Implementation of the Winning Version
Once the experiment is approved, the live version can be implemented on the site.

Main post- Role of Experiments in growth marketing

DEV Community: .

How does A/B testing work?

The Role of A/B Test Experiments in Growth Marketing