Multivariate Ad Optimization and A/B Testing, Can You Tell the Siblings Apart?


A/B, multivariate, both or neither? Try your hand at the questions below.

  1. Which method tests your media plan?

  2. Which testing method isn’t testing optimal performance, and instead focused on campaign insights?

  3. Which is the best testing method for scaling creative optimization?

  4. Which sibling requires you to pre-specify creative variants to test?

  5. Which sibling is available in Aarki Encore?


  1. Neither. The context in which we are discussing A/B and multivariate testing is as it relates to the creative. Creative testing and optimization is the process of comparing different variants to identify the variant that is likely to deliver the best results for a particular situation.

  1. A/B Testing. The goal of A/B testing isn’t necessarily optimal performance, but rather to provide campaign insight. For instance, if you are interested to learn which call-to-action is generally more effective within the context of same creative and similar audience, A/B testing can be a great way to find out. You can run an experiment with a fairly small sample, and by the end of it you might learn something that could prove valuable. Multivariate testing uses visitor behavior to not only determine the best creative variant, but also how important the role of each element is in the creative performance.

  1. Multivariate. Scaling manual A/B testing for optimization purposes is not an easy task. While, in general, one call-to-action “A” might perform better than call-to-action “B,” it is not a universal finding. It might be that a subset of your audience actually prefers “B” or your audience preferences may change with time. To account for these variations, you would need to setup and run new A/B tests on segments within your audience and adjust the “winner” daily. This is an unwieldy task, which becomes impossible as the number of variations increase. Multivariate uses advanced machine learning to develop a model for the creative and continually adjusts model parameters to find a dynamically optimal solution.

  1. A/B Testing. You have to specify which creative elements,  such as video, audio, background image, ad copy, layout, button color etc., you would like to test, whereas multivariate will automatically test all elements and optimize them based on results. This also means A/B testing allows for a limited number of variants to be tested at the same time, multivariate which can test an unlimited number.

  1. Both. We recognize the importance of both A/B testing and multivariate for optimizing creative, which is why both are included in Aarki Encore. Aarki Encore is currently the only platform in the industry that allows creative optimization of mobile ads using a true multivariate algorithm. It allows simple A/B tests if that is all you want. But it also enables dynamically optimized campaigns where thousands of creative variants are shuffled by complex algorithms using dynamic data coming from a variety of feeds and user behavior. As a result, you are assured of the best ad performance under all circumstances.

How did you fare? Will you be able to tell the siblings apart? If you scored poorly, we suggest you read our white paper, “Multivariate Creative Optimization of Mobile Ads,” and blog post “Multivariate, Give the People What They Want.

shutterstock_130332260“Well hello there mobile advertising,” said multivariate to A/B testing the fine morning it was born in Aarki Encore. If you haven’t heard, we announced the launch of multivariate creative optimization in our mobile ad platform.

While excited about the new baby in the family, we haven’t totally abandoned its brother – A/B testing. There is a time and a place for everything, including an uncomfortable amount of personification in a blog post about mobile advertising creative optimization.

Like most siblings, the two methods of testing are quite different, take the quiz below to see how you fare on telling them apart. Careful, you don’t want to offend one calling it by the wrong name.




Topics: Creative Insights