The Gutenberg Rule

Recently a couple of people have reminded me about how we'd used this design principle during MVT testing and yielded some good results and insights as a result. So I thought I'd commit some learnings to a post on the subject .


The Gutenberg Rule is a design philosophy named after the designer of the printing press Johannes Gutenberg. This principle suggests that people read content top to bottom and left to right.  You can therefore split a page into four quadrants, the “Primary Optical Area” in top-left, the “Strong Fallow Area” in top-right, the “Weak Fallow Area” in the bottom-left and a “Terminal Area” in bottom-right. Splitting a web page into four quadrants as illustrated below we tested the various positions of a product offer by rotating it through these 4 positions (in more than 1 test). 

This testing confirmed that

• position 1 yielded the highest uplift
• position 2 the second highest
• position 3 the third most profitable position
• position 4 the least uplift

• Additionally, just below position 2 proves to be the ideal location to place a Call To Action in numerous optimisation exercises

Horizontal Positioning

Extending on from this principle it's also worth noting that horizontal positioning is of equal significance, born out with the following test example. On a landing page we rotated three product benefits through a horizontal layout as follows and monitored the effects on click to apply rate. Swapping the 'Great rate' benefit to second position after a cash back offer yielded a 3.24% uplift.

Again, swapping the Overdraft benefit with the Cash back tile yielded an even greater uplift of 3.69%. I guess you could call this the "Gutenberg Horizontal Positioning rule".

So in conclusion, positioning of message and offers can be absolutely crucial to the success or failure of a web design based upon some highly established design principles.

Good old button testing

I

Everyone, at some point, does some optimisation testing of buttons designs. Some people think it's a trivial exercise to undertake when there's bigger fish to fry. Well I disagree, button design testing is exactly the kind of thing you can be doing quickly and easily with Google Optimiser or similar. We've done loads of testing in the past on buttons, testing colours, sizes, Apply text and so on,  but I read an interesting article from Get Elastic on how unusual button designs can give you an easy uplift in conversion. So I tried over the course of a couple of months on a landing page testing all the designs you see here. No.1 was the default design, and the winner was...No.3 the 'boxed arrow' design with a 32% uplift in click to apply rate. The arrow-based designs were in the upper end of the winning designs overall, but the, *cough* phallus-based designs stole an early lead but didn't win out overall. Give it a go on your website, it's quick easy and surprisingly fun.

Google Experiments Follow-Up experiments

First off - What is a follow-up experiment?

Google says:

"

When the results of an experiment suggest a winning combination, you can choose to stop that experiment and run another where the only two combinations are the original and the winning combination. The winning combination will get most of the traffic while the original gets the remaining. This way, you can effectively install the winner and check to see how it performs against the original to verify your previous results."

And why should I run a follow-up experiment?

"Running a follow-up experiment will give you two benefits. First, it will enable you to verify the results of your original experiment by running a winning combination alongside the original. Second, it will maximize conversions, by delivering the winning combination to the majority of your users. We encourage you to run follow-up experiments to get the best, most confident results for any changes you make to your site."

But what happens when a follow-up experiment delivers contradictory results?

 The screenshot below shows the original MVT test results....

 I commenced a follow-up test running the the winning variant from this test in a head to head with the original default. And this is what happened...

The blue line is the original design beating the first test winning variant. This has happened time & again with my follow-up experiments. Then I noticed something. When you set up a follow-up experiment it's easy to overlook the weightings setting or the 'choose the percentage of visitors that will see your selected combination' option of a follow-up test. By default it's set to 95% for your selected combination.

Now I cant offer any explanation but from previous testing with other tools such as Maxymiser we've seen when you up-weight a particular variant in a test in favour of another, invariably it's conversion performance goes down, sometimes radically so. I recommend only doing a 50-50 weighting at anytime in any follow-up experiment because for whatever reason an unequal weighting seems to skew performance.  Just be aware of this possibility and you'll be fine : )

If anyone can offer me a scientific explanation for this behaviour I'm all ears!

By the way, below shows the test after the weightings are reset to a 50/50 split. Bit different from the original follow-up experment no?

Give a Huq?

It's good to see some examples of AB testing that aren't just about which web page works best. And here's another example of Magazine cover split testing. This months issue of Company magazine is running with two variations of it's cover, one with presenter Fearne Cotton, the other with presenter Konnie Huq (who's married to this guy by the way).  As you can see both covers are the same bar mention of the featured presenter and the hero shot; even the poses are almost identical. I've mentioned magazine cover testing before in a previous article here but I havn't seen any recent evidence of the Press engaging in this kind of marketing test in recent years. Would be (mildly) interesting to see who wins this particular test.

Google Trends for MVT Terminology

Out of curiousity I've ran a couple of queries in Google Trends to see what are the more popular terms in the world of web optimisation. I queried 'ab test' versus 'split test' and found the latter to be less widely used. However I was more keen to see exactly what terms people were using for 'MVT', 'multivariate',   versus the old skool 'multivariable'. I thought that the term MVT would be more widely used in this day & age, or indeed be on the increase. It turns out to be far less popular than multivariate and on a par with 'multivariable'. I think it would be handy if we all stuck to a single expression, my personal preference being 'MVT'! This would certainly help for job searchs too! This particular blog gets ranked well for the term 'MVT blog' but is off the radar for 'Multivariate blog' too, so there's another side-effect to of our mis-aligned industry terminolgy! : )

UPDATE 16th August 2011: Whilst Google Trends is still a good tool for examining terminology usage or just trending full stop a far better tool for this job is Google's Keyword Tool . Obvious really : )