Riding the tsunami - optimising an online marketing campaign

In the last three months we've taken web optimisation in a new direction. To date we've been trying, (emphasis on 'trying') to follow the essential principles of mvt and split testing. Putting in test content, wait a period of time, wait for visitor volume to do it's work, seeing what works in getting more customers to convert, seeing what doesn't, etc, etc. This is all well and good and will always be the core of what we do, but recently I've decided to just go hell for leather on getting an uplift and cashing in on an already positive proposition.

Our bank pioneered the cash switching incentive. This is where you can switch current accounts and the bank (us) will give you £100 cashback for the pleasure. This has been an extremly good promotion for us and we will routinely serve this promotion on the website to drive current account applications, and boy does it work. I cant give you the monetary figures for obvious reasons, but there's a very good reason other highstreet banks are now offering this same incentive. Anyway, after seeing the sudden peak in traffic that you get to the site during the campaign I decided to perform an optimisation test during the promotion period on the customer journey or offer page. I basicaly came up with half a dozen alternative page designs that focus on the promotion and were ran directly against the default page design using our normal testing tool (Maxymiser). Because of the nature of the promotion period it is definitly not a conventional MVT or split test. I had no intention of seeing what would emerge as the outright winner over time I just wanted to serve the best performing content for that moment in time. So you run the 'test' and due to the massive traffic volumes you can just sit there and watch the customers come in and crank through the test variants very quickly. You soon see what works and what doesn't. I then engage in some aggressive culling of the negative performing variants.

So after an initial trial I already know what the best page design is for this particular campaign and will serve this content again during the next campain period, knowing it will be sucessful and do it's job. This means I can just challenge this 'champion' at my leisure. But in terms of results the first run gave me a 44% uplift over the default page design in getting people to apply for the product. Wow!

UPDATE: 15th August 2011.
Coming late to the party, but getting there in the end are Get Elastic ( a very good web optimisation site) endorsing this approach (after a bit of soul searching) with an article titled 

Should You Avoid Testing During a Traffic Spike?

Optimization of a product page

Like most highstreet banks we have a Loans product page, but we hadn't carried out any kind of optimization activity on this page, or this product for that matter. Generally speaking the thing that sells a loan is the rate (natch). But we have found that in terms of what visitors want to see on a Loans product page is the ability to use a loans calculator to see what their repayments are likely to be etc.
Therefore the following test was planned and conducted...

Objective – To get more people to use the Loans Calculator and more people to apply for the Loan product and submit the application form.

The test was ran continuously for 27 days between the dates of 21/08/2009 to 16/09/2009.

Test audience – 100% of traffic to Highstreet Bank Loans Hub page.

Hypothesis – By testing alternative page designs on the Loans Hub page we expected to see an improvement in the visitor to apply ratio by 3% and improve our visitor to submitted application ratio by 5%. We also hoped to see an improvement in visitors using the calculator by 3%

Test Results Summary

• 9.08% improvement in visitor to apply ratio
• 4.59% improvement in the submitted application ratio
• 4.12% improvement in visitors using the calculator
• 5.06% improvement in product accept rate

The existing page design was as follows (note the placement of the loans calculator)...

The winning page design was as follows...



Put into context
To put this improvement into context, if all of the 126439 visitors during the test period had been shown the new design :

126439 Visitors to winning design x 16.17% visitor conversion x 33.2% accept rate during test = 6787 accepted applications

126439 Visitors to default design x 15.17% visitor conversion x 31.6% accept rate during test = 6061 accepted applications

This equates to a 11.9% increase in accepted applications

Findings

Bringing the loan calculator and apply now button front and centre focuses a visitor on the primary calls to action; perform a calculation, click apply.

The static right-hand promo tile whilst previously used to host the loan calculator was now used to promote creative that played on 3 core themes; ‘cash’, ‘car’ and ‘home improvement’. We wanted to see which image and theme appealed to the most loans customers. In the past we have used a variety of images in the loans section that appealed to these areas, this test would go some way in answering which theme had the widest appeal.

We’re aware that images can be largely subjective in either turning people on or off but using a very basic image that illustrated a single concept provides a jump-off point for enhancing the ‘home improvement’ message relating to a personal loan application.

How to increase average spend - The 97p Shop

Cross selling and add-on sales are nothing new. If you visit Amazon often enough you'll be more than aware of the best in class method of cross selling, they are the masters. Wish lists, gift lists, recommends, you name it, they have it. Anyway something caught my eye the other day when I was visiting wilkinsonplus.com. They've introduced the 97p shop which essentially offers a range of items for, yes you've guessed it 97p. They promote this shop right across the customer journey.

1.



2.



3.





By the time you've selected an item from their main range and you are at the checkout your already primed to top-up your purchase with an item from the seemingly low cost range. Most people wont blink at spending an extra 97p on the average transaction and this particular price promotion will easily fly under most shoppers cost radar. The growth and success of retailers like Pound Stretcher on the UK highstreet during this economic downturn also adds weight to this particular promotion. We've already adjusted our psychological perceptions of the value of a £1 and how far it should go. Wilkinsons are effectively taking on these established budget shops by playing them at their own game. A similar approach can be seen if you visit your local Tescos where they've introduced budget ranges that compete directly with the likes of Lidl and Aldi. If Wilkinson.com make a success of this campaign (as I suspect they might) they could end up effectively owning the 97p value in customers minds and will be an endemic part of their brand

Eye Tracking

I recently had the opportunity to witness Eye Tracking research on our websites. This was undertaken at a company called Foolproof in London. Eye tracking is the process of measuring either the point of gaze ("where we are looking") or the motion of an eye relative to the head. Eye tracking relating to website usage is a method of capturing eye movement relating to web page design, content, layout and it's usage. Eye trackers are used in research on the visual system, in psychology, in cognitive linguistics and in product design.

With regards to this particular session you get to sit in a viewing suite, on comfy sofas, while people next door are monitored whilst they conduct a customer journey on your site. Their eye movements are tracked and you can see their monitor in front of you projected onto a big screen. The image below is of this event captured rather badly on my mobile camera (apologies for the terrible quality) Bottom left is the interviewee on a video feed and on the right is a projection of their monitor with eye tracking overlaid in realtime.




It's all very interesting. You do get a real insite into what people genuinely experience in the real world. There were many learnings to be had from this session. Too many to repeat here. One key one for me however, was how users view our '3 Box Wonder page' design. This is a page design we arrived at as an outcome of extensive multivariate tests (see image below). It's an unpopular design here in ecommerce but it performs consistently well in getting customers to convert. The eye tracking finding suggested that the reason it may be successful is that the negative points relating to the product are contained within the 3 stacked boxes to the right. These boxes happen to look like Google adword ads so are almost subconsiously ignored by the visitor. This is not something we had intended by this design but it certainly goes some way to explain why this design is so good at what it does! It also helps to prove the value of eye tracking too.

3Box Wonder...

Twitter tools (Part Two) - revised

Following on from my last Twitter related post about a few tools that caught my eye I'm adding a few more to the list (maybe not all 100% useful but show an interesting use of the Twitter API none-the-less):





1. http://tweetingtoohard.com/ - This site describes itself as a place 'Where self-important tweets get the recognition they deserve'. It's a listing of the most egotistical tweets/tweeters out there. Disturbing but fun.





2. http://tweleted.com/ - Thought that tweet had been deleted? Think again. Handy for looking up Gail Porters (@Gailporter) mobile number too.





3. http://twitturly.com/ - This site tracks and ranks what URLs people are talking about on Twitter. Looking up individual Tweeters URL postings is quite useful too.










4. http://dossy.org/twitter/karma/ - Similar to my previous recommendation of http://friendorfollow.com/ for graphically representing who's following who but in a much slicker interface.









5. http://twistori.com/ - A failrly basic, but slightly compulsive site for displaying live feeds, grouped by six simple concepts; love, hate, believe, feel and wish.



Read part one of this post (more Twitter tools)