MailChimp

Multivariate testing

My Role

Product Designer

The Challenge

A/B testing was a really popular request from customers. They wanted the ability to compare more than 3 emails. They also wanted the ability to test more than a single design element.

The Solution

Our solution was to allow them to test up to 8 different versions of content with Multivariate Testing. For example, users could use Multivariate to test 2 subject lines and 2 from names to compare the performance of 4 different campaigns. After a test was created, a report was generated that let the user compare the results of each combination and the results of each variable.

Early sketch for the content comparison feature in a multivariate test.
Initial sketch for the multivariate report.
The layout for variables step in a multivariate test.
Various sketches for the variant selection interface in a multivariate test.
In the final version of the variable page, the user selects their variants along with how they would want to test the combinations. As selections are made in the left column, the right column explains how the user's list would be affected.
Users would setup their multivariate test from this page.
Users would add content to each campaign in their multivariate test from this page.
The final version of the content comparison page for a multivariate test allowed users to hover over the rows in the left column and see a preview of the content in the right column.
The final version of a report from a multivariate test.
© Copyright 2021, Tyrick Christian
hello@tyrick.com