What is MVT testing?

In it's simplest form MVT or multivariate testing is where you test alternative experiences of an existing webpage against the current page design or user journey. Unlike AB or Split testing where you test one webpage design against another, in MVT you test a combination of page elements like alternative page copy, headings, images, buttons and so on all at once to see which specific combination of alternative designs work best in driving page visitors or customers towards an end goal of your choosing.

All testing, regardless of whether its an AB or MVT test requires two things. Time and Traffic.  By that I mean you need to give a test enough time to establish what's called statistical significance, to basically reach a stable conclusion, and you will also require a volume of website traffic to churn through your test experiences. Typically you need less traffic to run through an AB test which normally only has a couple of alternative page designs to get through. In an MVT test you will typically have a higher number of test combinations to get through and therefore will require enough traffic to 'feed' your test.

In all test situations you continue to send a proportion of traffic to your existing page or default page to benchmark your alternative experiences against it. This is where you are calculationg any uplift in conversion, be that click through rate or the number of people getting to and passing through your check out process.

Testing outcomes can be positive or negative. A positive outcome might be that you test a red check out button versus a blue check out button and the red button delivers 8% more purchases. A negative outcome might be that the same button delivers a -5% drop in purchases. However, either way you have a valuable learning that tells you what changes to make to your site or inform further testing.

Calculating Uplift

This is fairly basic but a common query none-the-less:

Q. How do you calculate conversion uplift?
A. Winning% - Old% / Old% x 100 = UPLIFT%

Simple!