Marketo and other systems provide options for determining the most effective marketing content. Companies can use a multitude of tools, including A/B testing and Champion/Challenger testing, to optimize such content.
During A/B testing, a company tests two different versions of a product, which often consist of emails or web pages, to see which one has more success. The “A” version is that which is expected to heed the best results, and the “B” version is expected to be the next best option. A/B testing employs each of the two versions to see which receives a better response, with half of the test subjects receiving the “A” version and the other half receiving the “B” version. A/B testing consists of a single test to determine the winner.
Champion/Challenger testing is continuous until a company determines a winner (1). With Champion/Challenger testing, the “Champion” version is the version thought to be likely to produce the best results, and the “Challenger” version is the version thought to be the second choice option. Thus, after Champion/Challenger testing is conducted, the “Champion” that is decided upon is utilized in a company’s marketing efforts.
Within the Marketo marketing automation system, companies can conduct both A/B testing and Champion/Challenger testing by sending the “A” or the “Champion” email to one group of people, and sending the “B” or “Challenger” email to another group. Companies can choose which criteria decide the winner of this testing, which can include “Opens,” “Clicks,” “Click to Open %,” “Engagement Score,” and “Customer Conversion” (2). Marketo makes testing straightforward, so that, through Marketo, a company can input the two versions it wants to test, send the emails, and find the results that they need.
When interpreting the results of A/B testing and Champion/Challenger testing, the winner is chosen if the results are statistically significant. If the “A” or the “Champion” option produces similar results to the “B” or “Challenger” option, causing the results to be statistically insignificant, then a company may need to test more campaigns to attain better results. If the testing consistently has statistically insignificant results, then the two options may simply produce similar outcomes. There are many free online statistical calculators available, including the AB Testguide, that companies can utilize to detect whether their testing had results that were statistically significant (3).
RightWave provides expertise and guidance to its customers for marketing strategies and tactics. For more information, contact RightWave.
Sources:
- Marketo, “Champion/Challenger vs A/B Test in Engagement Program”, 25 September 2019, https://nation.marketo.com/t5/Product-Discussions/Champion-Challenger-vs-A-B-Test-in-Engagement-Program/td-p/193849
- Marketo, “Creating an A/B Test program in Marketo”, Undated, https://nation.marketo.com/t5/Product-Documents/Creating-an-A-B-Test-program-in-Marketo/ta-p/248391
- AB Testguide, “AB Testguide”, Copyright 2018, https://abtestguide.com/calc/
