Friday, 30 April 2010

An unbiased review of Google Experiments

I notice that a lot of reviews for Google Website Optimizer (GWO) have been done by what at first appear to be independent companies & individuals, but at second glance are actually affiliated or partnered to Google in some way. So this being the case, here's my unbiased opinion & experience of GWO for what it's worth. And please note: I am not pushing or offering any service here relating to Google.

Anyway, I've been itching to give GWO a trial for the past couple of years. Up until now our websites have not been conducive to implementing Google tags and besides which we've been using a managed testing service from Maxymiser. However a few things attract me towards GWO over a paid service.

  • First off, it's free. If you have the means to implement Google tracking tags into your web pages, why wouldn't you at least try a test or two?
  • Secondly, even if like our company your paying a third party company to build and run multivariate and AB tests for you on your behalf, no matter what contract you're on there's never enough time and resource to run with all testing concepts and ideas you might like. Which is why I've persisted with GWO for the past couple of months as it's relieved a bottleneck in our testing schedule being able to run quick & dirty tests on the fly alongside our more formal testing.
  • If your business is to be an optimization expert you need to try a variety of tools, especially the most commonly used one to add to your knowledge base.

We now have a sub domain now where we can do whatever we like on landing pages including testing and tagging with GWO. I've now conducted three landing page tests ranging from AB tests to MVT and just launched my fourth.
  1. My first test was on a Personal Loans page where we tried a dozen different page designs and copy changes to see if we could get more people to apply for a loan. It was a fairly simplistic affair but has yielded a 14.77% uplift in visitor conversion by offering up a different page header and product introduction copy.
  2. The second test was on a High Interest Current Account  landing page (phase 1). This looked at simplifying overly complicated product information and has yielded a 20% uplift in conversion. I've just launched a phase 2 of this test where I champion/challenge the previous winner, a rather stodgy design against a more creative led design.
  3. My third test was recreating a Bank Accounts comparison page from our main website but in a landing page environment. We then employed the usability web experts Foolproof to come up with an improved design for the page and test it (amongst other designs) against the default page. It's still running and yielding a 9% uplift in submitted current account applications to date.

Now  the problems with GWO.

For whatever reason, aside from my most recent test, none of the tests have displayed progress data in the reporting interface of the GWO console! Fortunately in every test I have been able to tag each variant with a bespoke/unique tracking value which appears in our downstream sales database when someone submits an application . Because of this I have been able run successful tests with a reliable MI to go by.

Customer support for GWO is non-existent and you're reliable on forums to get any kind of useful information about troubleshooting. I have had to solve a number of glitches with the tests ran to date myself using significant time & effort!

Because I have web developer experience I am able to setup a test and build the page variants for my test also. But this is not the norm and I can imagine working with an internal or external agency to implement and update GWO test tags and content might make the whole experience unworkable.

Actually coding the test variants is badly handled in that they just give you a single unformatted text area to do your HTML editing in, so lot's of cutting and pasting from a conventional web editing tool is involved, such as in Dream Weaver  or Visual Studio (even Notepad is better!).

The positives of GWO 

I love the GWO console. Both thr reporting interface (below is the reporting interface showing a current MVT test in progress) and the test build interface are simplistic but highly functional in design. The step by step code implementation are really straight forward. You tell GWO what your original, test and conversion pages are and it tells you what code to paste into those pages to get the test going.

It allows you to weight test variants so that they are displayed to a set portion of your test audience, and the test follow-up feature is great too for validating the results of a concluded test.

If Google could come up with a solution that didn't involve placing tags on test and conversion pages then I think the technology could take off like a rocket.

In conclusion

So besides some teething problems with getting up to speed with GWO its been a good learning curve and I've been able to turn in some very reasonable test results to compliment my other testing.

If you want  to see a previous article comparing GWO to other optimisation providers click here