A number of things have changed in the world of MVT testing, AB testing and Conversion Rate Optimization (CRO) generally over the last few years. The question we have to ask of ourselves and the people we work with is whether we have truly kept pace with these changes. I've asked this question a lot recently and a number of areas have been found wanting.1. The economic collapse of 2008 changed everything both in terms of the economy but also the web, how eCommerce businesses either thrived or died. The reduction of cash flow in internet businesses both drove the argument for testing and improving your online experience to squeeze the very last drop of our traffic and it also threw greater intensity on the need to prove ROI . If CRO was ever seen as peripheral then this was the time to prove your worth and justify the ongoing investment as a core player in the online channel not waiting in the wings. Verdict: Done OK. Could do better.
2. Web traffic has radically shifted into different segments. The Mobile and Tablet device accounts for around 30% of web traffic at the moment and continues to rise. As a consequence traditional fixed web testing is becoming less and less relevant as each day passes. If you're not actively testing in these new segments now it may already be too late. Despite some early and ongoing dabbling in these emerging segments we have failed to make any real inroads in terms of replicating the successes achieved in desktop testing. Verdict: Strong start but faded fast.
3. Are still looking at the same things to test time and again? Sometimes you need to retest the same concepts and experiences especially those user journeys core to your sales and service funnel but sometimes you need a fresh pair of eyes on your business to get new insight and ideas. Verdict: Good but room for improvement.
4. Your choice of testing vendor is still key. The things you look for in a testing partner are those that can help address the shortcomings mentioned above. Someone who does enable testing in those emerging segments, who continually helps evidence ROI for your CRO activity to the budget holders, and someone who is proactive and brings fresh ideas and insight to the table. More importantly now you need someone who wants to be embedded in your business and add value day in day out and doesn't get complacent. Verdict: TBC
I firmly believe that MVT and CRO can still offer huge uptick in web performance for the online player. I guess the biggest danger is falling into patterns of working and thinking that worked well in the past but don't necessarily work as well today. Continual self assessment and re-evaluation of all the components in your testing programme should still yield success but you might have to make some tough decisions and change to bring about renewed success.
