Intelligent Enterprise

Better Insight for Business Decisions

RSS
Webcasts
Digital Library
Subscribe
Home

February 1, 2003

Actionable E-Metrics

An actionable online analytics framework is a key ingredient in any intelligent enterprise

by Allen S. Crane

Many of the reasons offered to explain the dot-com meltdown involve financial analyses of the economic models used — models that would supposedly have become profitable with more venture capitalist spending.

While poor business plans were certainly the root cause for a number of over-committed companies, such blanket explanations don't fully describe a true cause and effect. Many successful dot-coms — such as Yahoo, Amazon, and eBay — continue to do business, despite their "if we spend first, they will come" business approach.

Given similar spend-first strategies, why do some e-businesses fail, while others prosper? In essence, some fail because they are unable to develop a new set of metrics that differs fundamentally from traditional finance-based metrics, yet is intrinsic to e-commerce. Too few companies understand that the Internet is not simply a "channel" for business; it redefines the business itself. The nontraditional business frontier demands nontraditional metrics to manage it.

In this article, I won't attempt to redefine the industry of e-commerce measurement tools, nor will I claim to offer a silver-bullet cure for struggling e-businesses. Rather, I'll present an original framework for how to measure and analyze real-world, relevant, e-commerce metrics, and how to present them in an actionable, executable format. Ultimately, this framework gives you the ability to:

Rapidly diagnose site problems
Understand the elements of site conversion
Understand the relationships among units, revenue, and margin, and the behaviors that drive them.

Traditional and "Blue Sky" Metrics

Traditional profit and loss (P&L)-based metrics fall far short of painting a complete customer behavior picture. Such metrics are essentially transaction-based — they're gathered only at the time of sale. In contrast, metrics for e-commerce (hereafter referred to as e-metrics or clickstream metrics) offer the opportunity to view more than the final transaction of a purchase. The beauty of online data is that it captures every click visitors make and every image they see. Clickstream data offers a rich picture of all the customer-behavior events that lead to a purchase (or, perhaps more important, the events that result in a nonpurchase).

During the dot-com explosion, several e-metrics buzzwords infiltrated the analyst vernacular: page leakage, stickiness, slipperiness, velocity, shopping cart abandonment, convergence, and perhaps the most seductive of them all, path analysis (the ability to find the "magic path to purchase").

These concepts are interesting, but they tend to be more of an academic exercise than anything else because they're based on anecdotal scenarios. A former director of mine perhaps said it best when he asked me, "So what if a page is sticky? Is that a factor of the page confusing customers who don't know where to go, or are they investigating our products, and their time spent is proportional to their interest? How can we tell the difference?"

To get to an actionable decision, the most effective approach lies somewhere between P&L metrics (which are vague and not properly integrated) and the academic metrics I've described (which can be interesting but nonactionable). The Holy Grail is a suite of quantifiable, actionable e-metrics that capture behavior patterns and accurately relate them to the key transactional business levers of units, revenue, and margin. Ideally, the reporting of all such data should be managed and developed together, and accessed by flexible reporting tools that can accommodate technical as well as nontechnical users. Ideally, the entire process should be managed by an ultra-lean, experienced support staff that balances tactical and strategic visions.

Where to Begin

Consider the questions that you're trying to answer about your Web site visitors and you'll find that they range widely. The first step toward building a successful analysis toolset is to broadly categorize these types of analyses. While each analysis is unique, most analytic questions can be framed in a two-by-two matrix (see Figure 1). I'll examine each part of this matrix in more detail.

Quality e-metrics. These basic traffic metrics are known for their short-term/low integration effort. They rely only on clickstream data and report on a predefined set of data. At the minimum, such reports must be available on a weekly basis, but the code must also be readily available to query the data directly as needed, if the project can't wait until the week's data is ready. Examples include:

Traffic by page and site area (groups of pages): the raw traffic data in terms of requests (clicks), visits (per session-based logic), and users (per cookie or other individual or machine-specific identifier).
Page leakage percentage: the percentage of the number of visits to any particular page where that page was the last page in a visit, divided by the number of visits to that page. By identifying pages that result in a high percentage of visit termination, you can better address site design, either by intuitive means or qualitative usability testing. In either case, before and after results can be compared in order to identify acceptable limits for leakage on certain pages.
Next click: a component of path analysis, the pages that most frequently followed in direct sequence from a page or a set of pages. (Because of the number of enumerations made possible by Web site design, you should first try this metric on a small subset of pages.) Next click is critical for site design, when quantifying navigation patterns from certain main pages that contain a multitude of links. Even more importantly, it helps you quantify of one of the most elusive metrics of Web design: site real estate value.
Previous click: similar to next click, a previous click report will yield the pages that most frequently occurred directly prior to a page or a set of pages. (My previous advice to try this metric on a small subset of pages applies here as well.) Navigation patterns and site real estate are key insights from this metric. On a related note, referring domain and referring URL reporting can be extremely useful, especially when presented in a format that illustrates trending effects over time.

It's important to note that quality e-metrics are fully automated and published regularly. Minimal effort should be spent in maintaining such regular reporting metrics.

Project-centric metrics. Although they rely solely on clickstream data, these metrics are designed to be more long-term in scope — long-term meaning that these analyses are designed to facilitate online application processes that will result in measurable technological changes. Such metrics are designed to facilitate the transition to better processes (strategic in nature) by measuring the before and after cases.

For example, if a project manager wants to improve a part of the Web site, the Web analytics consultant would work with the project manager to create a set of metrics around the specific pages that need to be improved. These metrics could include frequency of visit — or, if the set of Web pages is systemic and follows a process (a registration or application process, for example) — the metrics could include step-by-step fallout detail, also known as a waterfall report.

Waterfall reports are invaluable for analyzing the effectiveness of the sales funnel, or the ratio of the number of customers in one stage of the sales process, to the number of customers at another stage in the sales process. Say, for example, that the project manager wants to move more customers online in order to reduce phone calls to the customer service center. The Web analytics consultant would create a metrics package that the project manager could run through the course of the implementation to see if more customers used the new and improved Web application as a percentage of all customers (Web and phone) who interact with the company.

Such metrics are necessary to sustain results and substantiate quantifiable financial benefit. When the project is completed and the business process improvement is booked to the financials, it may no longer be necessary to run such reporting — therefore the term disposable metrics. However, while the metrics may no longer be used in running the day-to-day business, it's important to note that the metrics were critical to proving the value of the project and lowering the overall cost-to-serve.

These metrics are essentially queries that are developed in close consultation with business users, specifically project managers. When the queries are developed, the project manager should be given the database access and reporting tools necessary to run the specific-use reports as needed.

Although these metrics are designed to facilitate the strategic implementation of technical projects, these queries are often disposable, in that they may become short-lived after a project's implementation. However, if after an implementation these metrics are deemed to be business imperatives, they're considered for regular publishing as quality e-metrics.

Deep dive metrics. These metrics are known for their short-term/high integration effort. They are 100-percent customized and thus among the most time-consuming metrics to develop, mainly because they're designed to answer any of a large range of questions. These questions may include root-cause analysis, hypothesis testing, developing confidence intervals, online customer cluster analysis, shopping cart (market basket) analysis, sequence and association analysis, and other data mining applications.

Deep dive metrics can be very difficult to obtain, for at least three reasons:

They require integration among multiple databases, which is almost never as straightforward as it sounds.
The complex data manipulation required to perform the relevant statistical analysis is often complicated by huge amounts of data typical of online clickstream analysis.
The statistical analysis is often a first of its kind, requiring technical "acid-tests" by peer or managerial reviewers.

It's a rare company indeed that contains clickstream, financial, customer, and support databases in a single, seamless repository on a single database platform. The reality is almost always complicated by multiple database instances running on multiple platforms, requiring extracts of data to other databases and matching on key fields to get additional data, which is then extracted to additional databases and further matched before final reporting.