Use the PIE Method to Prioritize Ecommerce Tests

via Practical Ecommerce http://www.practicalecommerce.com/articles/61922-Use-the-PIE-Method-to-Prioritize-Ecommerce-Tests

In “Top 5 Things to Test in Ecommerce Websites,” my previous article, I addressed the testing priorities that merchants should focus on. However, it’s not realistic to tackle all five tests at once, and depending on the unique circumstances around your site and business there can be many more areas to test.

Since every site is different, it’s not always easy to know where to start. After all, even if you have resources to plan and create tests, you may not have enough traffic to split between them.

Worse, without prioritization, you could spend months testing the wrong areas of your website, waiting too long for tests to complete, and missing the much more important areas. The opportunity cost is significant. So making the right choices is crucial.

How to Prioritize your Optimization Efforts?

This is a key part of “optimizing the optimization process” and it’s an important step to avoid wasting significant time and effort. At WiderFunnel, my optimization firm, before we start thinking about designing optimization tests for our clients, we always start with the prioritization process.

We use a simple tool we developed — the PIE Framework — to determine the best places to start. You can use the same approach to determine your own testing priorities for your ecommerce website.

The PIE Framework considers the three factors — “Potential,” “Importance,” and “Ease” — to prioritize which tests should get the most attention.

The PIE Framework helps you decide what is the best page to test.

1. Potential

How much improvement can be made on the pages? Although I’ve yet to find a page without some potential for improvement, you can’t test everywhere at once and you should prioritize your worst performers. This should take into account your web analytics data, customer data, and expert heuristic (or rule-of-thumb) analysis of user scenarios. Essentially, with “Potential” you’re ranking your pages based on how poorly they’re performing.

2. Importance

How valuable is the traffic to the pages? Your most important pages are the ones with the highest volume and the costliest traffic. You may have identified pages that perform terribly, but if they don’t have significant volume of costly traffic, they shouldn’t be testing priorities.

Your web analytics data will be one of your most valuable resources to prioritize the importance of each page. Your data will point to some of the clearly problematic pages — landing pages with high bounce rates, for instance — but some problem areas won’t be so obvious.

For example, if one problem you identify is a high rate of shopping-cart abandonment, your web analytics won’t tell you why people are abandoning their carts. Perhaps your visitors are only going to the shopping cart phase in an attempt to find shipping and returns information that should be shown earlier in the process. If you work on optimizing the cart, where web analytics points to the problem appearing, you won’t be able to fix the issue.

It’s always crucial when testing to step back from the website analytics data — i.e., visits, duration, and bounce rate — and remember to consider the financial aspect of the page, from a cost point of view as well as revenue. The cost of bringing visitors to your landing pages is an important consideration when prioritizing testing opportunities. Given a choice between testing on two pages with similar traffic and conversion problems, one with high-cost traffic sources will provide a better return on investment from your conversion-optimization efforts.

3. Ease

The third and final consideration is the degree of difficulty a test will take to get running on a page, which includes technical implementation as well as organizational or political barriers. How complicated will the test be to implement on the page or template? The less time and resources you need to invest for the same return, the better.

Don’t underestimate the “political” ease of a test opportunity: A page that would be technically easy may have many stakeholders or vested interests that can cause barriers.

In many organizations, you could add an arbitrary image to one of your product detail pages and no one in the organization would notice; but add a call-to-action button on the home page, and the CEO could call to ask if you’ve violated the company’s graphic standards policy.

You can quantify each of your potential opportunities based on these criteria to create your test priority list. We use the PIE Framework table to turn all of the data inputs into an objective number ranking.

You can use a PIE Framework table to determine which page has the most testing potential.

Your Prioritization Is Unique

There are no standard rules for which pages are best to prioritize. Your website lives in a unique target market, including factors like your competition, seasonality, and internal cultural environment.

All of these affect how your site is used and should be optimized. The priority rating you give each of your potential test pages will depend on this unique business environment.

View Data at the Page-template Level

I mentioned earlier that the home page can be a political hot potato in some organizations. The good news is that, despite appearances, your home page may not be the most important page on your site.

Even if your home page is the most popular single entrance page, it often loses its top ranking if you count all the pages that use the same template. Doing this will help to determine where you have opportunities to test site-wide template layouts in addition to individual pages.

You may get only a fraction of your entrance visits to any single one of your product detail pages compared to the home page. But if you add all the visits to all the product detail pages, there are probably many times as many entrances on these page as you get on your home page.

When WiderFunnel performs the PIE analysis for our clients, we’ll typically rate their standard template pages as a single page. This makes sense, since testing every individual product page in a large ecommerce site would mean thousands of tests and testing individual pages rather than the template creates an inconsistent experience for visitors, which can destroy any potential conversion rate gains. Template pages need to have a similar structure, or visitors will get distracted and confused as they move from product to product.

As you apply the PIE Framework to your ecommerce site, you should be able to prioritize your ecommerce tests in a logical way. You’ll remove gut feeling from the decision-making process and focus your team’s efforts on the right place at all times.