In today’s economic environment, it is critical to achieve a return on investment (ROI) for any budget that is spent. In the online environment, where the landscape changes so quickly—either due to competitive challenges or the increasing savvy and changing needs of the online consumer—achieving a ROI can be especially challenging. But the need to spend money online to remain relevant and compelling is an ongoing push.
Brands may regularly find themselves needing to do a full site redesign, or a partial site refresh, as part of their online strategic marketing plan. However, any investment in a website should be justified against the bottom line, making it necessary to demonstrate the quantifiable impact of any changes that are made to the online property.
So, What Might You Do?
There is a multi-phased (Pre-test/Post-test) approach, primarily used to measure the impact that changes have on a site (such as a full or partial redesign). The current website is first researched to gather benchmarking measurements. Keep in mind that by conducting research against the current website, real-time usability feedback and site experience data can be leveraged to guide the site redesign effort itself. The site is then updated (fully or partially revamped) and the new website is tested again.
Case Study
In a project for a client in the office supply category, Usability Sciences utilized this research design with dramatic results. The client brand team was gearing up for a major site redesign. In advance of this effort, Usability Sciences collaborated with the brand team on a two-phased research project that would serve two functions: 1) provide qualitative feedback on the current website that would direct the brand on where to focus their redesign efforts, and 2) provide quantitative benchmark measures of the performance of the site on key business indicators such as online conversion, success with site visit, and ease of use.
Phase I of the project, the pre-test, ran for six months. An online survey was utilized that included website entry and website exit questions to determine visit purpose and visit success, with various other demographic and key performance questions. Usability Sciences then conducted an analysis with the resulting findings becoming instrumental in focusing efforts on improving overall navigation and the checkout process. Redesign efforts continued and the new site was released three months later.
The following month, Phase II of the research project was launched when the post-test began by fielding the exact same survey on the new site, collecting data for the next four months. Upon conclusion, Usability Sciences conducted a second analysis, this time with the intention of comparing pre-test responses to post-test.
- Key Finding #1:The checkout process experienced a 28% increase in completed transactions. Changes to site checkout, such as shortening the number of steps in the process from start to finish, had a significant positive impact on the conversion rate for online transactions. By looking at checkout process data, we determined that the pre-test measure of successful completion of checkout was 25%, while the post-test measure of successful completion of checkout was 32%, an increase of 7 percentage points, or a 28% lift in productivity. By enabling visitors to more successfully navigate the checkout process, the transactional mission of the site is being met to a greater degree, and the redesigned site delivers online conversion at a quantifiably higher rate.
- Key Finding #2: Browsing navigation success was improved by 5%. Changes to site navigation, such as establishing a consistent page layout and implementing changes to the look and feel of the toolbar, had a significant positive impact on visitor browsing behaviors. By looking at visit experience data, we determined that the pre-test measure of the success rate of those seeking products using the browse path (as opposed to the search tool) was 79%, while the post-test measure of the success rate of those using the browse path was 83%, an increase of 4 percentage points, or a 5% lift in efficiency. By enabling visitors to more successfully match products to their needs, the organizational mission of the site is being met to a greater degree, and the redesigned site delivers a more powerful online branding experience at a quantifiably higher rate.
Parting Thoughts
In closing, as you face your own business needs of having to demonstrate return on investment with each of your online initiatives, a good rule of thumb to remember is that you can’t manage what you don’t measure. The Pre-test/Post-test methodology is an excellent tool to keep in your measurement toolbox for times when you are introducing a new online strategy (new look and feel, new messaging, usability enhancements) and you need to demonstrate the quantifiable impact of your approach.