Multivariate Testing: The Concept and Benefits

Multivariate Testing

Marketers are used to the fact that audience preferences are constantly changing. As users are swamped with more digital content, their expectations rapidly rise when selecting brands to engage with. The days of “one size fits all” or, in this case, “one ad fits all,” have ended. This challenges the mobile advertising industry to adapt methods, like multivariate testing, to target users with ads that align with their interests.

What is multivariate?

Multivariate is a creative optimization technology which allows testing multiple variants of ad elements to determine the most effective combination for a single ad. Rather than just changing one ad element, like a bee or a butterfly character as seen below, you can change a group of different elements to see which combinations perform better with your target users. The result would be the top performing ad, the “champion.”

Variant 1  Variant 2
Match3 Test Visual ENG

As the industry nowadays is shifting to highly personalized advertising to meet increasing user expectations, retargeting efforts can especially benefit from this process.

How to leverage multivariate testing for retargeting

Retargeting is already a powerful conversion optimization strategy, and to maximize its benefit you can unite it with multivariate testing to potentially deliver stronger results for your ad campaign. Some strategies to leverage multivariate to your retargeting campaigns are:

  • A/B Testing

A/B testing is creating different versions of an ad by tweaking a single element. This process helps advertisers gain valuable campaign insight to determine which variant works better than another. A/B testing for ads may include changing the character, booster, button color (as presented in the below ad creatives), and more.

Variant 1 Variant 2
Match3-RedCta Match3

A/B tests are more useful if you need meaningful results fast. However, it limits the number of variants you can test, unlike multivariate which can automatically test a number of elements and optimize the highest performing ad creative.
  • Localization

More advanced targeting capabilities, such as location data, now permit marketers to target customers locally, allowing a more specific, yet wider audience reach.


Localization should not be underestimated while working on creative variants. But localizing per country is not enough to get the most of your retargeting campaign. Serving an ad to a prospective user with the right language increases your chances of driving engagement.

Aarki leverages language localization to reach the users who speak the same language who live in different countries. Professional translators translate to 21 language variants and Aarki dynamically serves each variant with the main language spoken in each country.

Multivariate Case study

In order to determine the value of multivariate testing, we analyzed a programmatic advertising campaign for a casual game app over a period of four months.

We tested multivariate creative which showed 6 different backgrounds against the previous champion creative. During the initial phase of the campaign, there was no significant difference on the CPI and CTR performance between the champion and the multivariate creatives.

We continued the test and one by one removed variants that were underperforming and ran the performing variants against the champion with equal allocation. One of the backgrounds was able to outperform the champion in terms of the CTR (+3%), CPI (-25%) and CPP (-17%) performance. So we removed the previous champion and the best performing creative became the new champion creative.

This campaign proved that multivariate testing makes sure that your app marketing budget is spent on the right creative.

Aarki Encore, our proprietary programmatic advertising platform, is the sole platform in the industry that uses a true multivariate algorithm to allow creative optimization for mobile ads.

Topics: Creative Insights