The Gutenberg rule - revisited

In a previous post about the Gutenberg rule I mentioned that we'd achieved some good AB and MVT testing results using this design principle. The principle basically works off the theory that humans subconsciously scan a print or web page from top left to bottom right and then loop back up the page. So allowing this principle to manage your page layout seems to have become the norm in web design placing buttons and other key calls to action in the 'fertile' areas, typically bottom right.

Increasingly though I notice that more and more people are breaking from this practice whether through test learnings or just asthetic decisions. An example being Aviva.co.uk .As you can see below their landing page for car insurance aligns much of it's content to the left of the page. Having become comfortable with placing the onward journey point at the bottom right this just jars. Having said that it's important to challenge the norms to see what resonates with the customer. I may have to test this layout myself to check we're not missing a trick.

In the meantime here's a mock-up of an alternative design I would test based on established findings.

Happy testing : )

Page fold - browser size tool revisited

Problem: Recently Google have moved their Browser Size Tool into Google Analytics. For me and many other sites this means it simply no longer works as the 'In-page' reporting showing heatmaps and so on doesn't work or execute with our lovely CMS.Boo!
Solution: There's a free alternative. Yay! Called whereisthefold.com.

This is what it looks like on this blog.

a/b testing tools comparison by popularity

As part of my role I have to routinely compare A/B testing and MVT testing tools, suppliers and solutions. Many different sites list these suppliers but I've tried to use a few public metrics to determine how popular these tools are with the general public. I call it 'Reach ranking' in that suppliers are ordered 1). by their Alexa.com rankings, 2). then their average monthly searches for their test platform as determined by Google Trends and finally 3). how many clients they list (if at all) on their website.

It's also interesting to see so many new companies and start-ups there are in the CRO business since I last did this audit back in 2010 and also how many providers are no longer around. It's a fierce and competitive business....
So here's what I've found as at August 2014. Sources: Alexa.com, Google Adwords and Own Vendor sites
updated 13th Oct 2014

Reach ranking	Vendor	Alexa volumes YTD	Avg. monthly searches for product	No. of publically listed clients
1	SiteSpect	381,175	98,600	55
2	adlucent	324,719	27,000	0
3	Google Experiments	10,806,000	802,000	0
4	Conversion multiplier	3,987,218	73,600	0
5	Global Maxer	3,210,851	51,000	0
6	Conductrics	2,333,087	14,700	0
7	Qubit	1,506,519	25,880	131
8	Avenso	1,020,946	1,030	32
9	Clickthroo	974,000	47,819	0
10	Adobe Test and Target	945,671	78,000	0
11	Visual Website Optimizer	908,000	13,102	0
12	Accenture	860,000	13,240	0
13	Webtrends	763,000	19,600	0
14	Hi Conversion	722,004	47,000	44
15	Get Smart Content	673,173	3,740	14
16	Convert	672,000	126,000	0
17	Taplytics	283,016	12,789	6
18	Optimizely	239,000	58,898	25
19	Maxymiser	149,697	6,980	78
20	Site Tuners	121,678	13,000	72
21	Autonomy	120,271	26,000	0
22	AB Tasty	93,689	27,000	31
23	Monetate	36,501	21,230	39
24	Unbounce	22,230	12,230	0
25	Hubspot	2,700	541	0
26	Genetify	0	6,830	0
27	Vanity	0	6,400	0

Happy testing : )

What's the process for deciding what to test?

I constantly have to remind myself of the process I should go through in identifying what I need to test, the best use of testing resource and how to get buy in. To this end I've done a mildly influenced by Mad Men presentation here:

P.S The online presentation tool I used was Emaze

The ethics & morality of A/B testing - #FacebookExperiment

I was recently sent this article from Techcrunch on the 'Morality of A/B testing'. In brief the article criticises Facebook for recently conducting an A/B test of its newsfeed.It manipulated a subset of it's users newsfeed so that some were shown a higher proportion of negative stories over positive to see if that would impact on these same users on posts; i.e. would they post more negative items as a consequence? It sounds like the outcome was yes, people were influenced to be more negative when exposed to negative news. Techcrunch however questions the morality and ethics of conducting such a test in the first place and that users should at least be offered an opt out citing that people with emotional issues had been dangerously exposed to this unwitting exercise. OK so you cannot lump all A/B testing exercises into the same boat with this one. Conventionally testing has been used to achieve a commercial or service outcome not an emotional change. Perhaps people are right that Facebook shouldn't take it upon themselves to manipulate peoples experiences in this way for their own ends. However a sense or perspective is needed here. It's Facebook we're talking about here after all. If your personality is so fragile that you need to be shielded from any potential bad experience, negative page content in this instance, should you even be on the internet in the first place? To me this is a veiled stab at big companies use of our data, which is fair enough in some instances where trust has been abused. I resent optimisation testing being used in this way, painted as some dark art that should not be deployed without peoples full consent. Secondly it smacks of the Nanny state. Why do we think we should cosset and protect adults in this way? I worry that if this starts to go down the whole cookie directive opt in/out route we'll damage online testing and experimentation and inhibit a whole industry in the process. If we start telling people that they are part of a test group or experiment online and that they can opt out of it you're going to compromise the fundamentals of testing for want of a disproportionate response to an innocuous practice. UPDATE: 3rd July 2014. Looks like the Facebook study has become a bigger issue now with the BBC now covering it here. UPDATE: 29th July 2014. This just in from the BBC, OK Cupid admit testing their subscribers to see if connects can be made with people not normally seen as a good match.