November 15, 2002

Outward Bound

Why proactive business intelligence is a hallmark of the real-time enterprise

by Justin Langseth & Nithi Vivatrat

During the height of the Internet bubble, wireless technologies were near the top of every company's list of strategic priorities. Now that the bubble has burst, much of the hype surrounding wireless technologies has dissipated. However, there are some very real and compelling applications of wireless technologies in the business intelligence (BI) space. These applications are in large part a product of a major revolution in the BI world — the move from reactive to proactive BI.

The Proactive BI Revolution

The term proactive BI relates to a collection of technical and process innovations across the data warehousing and BI space, which together represent a major step forward for the BI world. Proactive BI focuses on decision-making acceleration by leveraging existing BI infrastructure to identify, calculate, and distribute up-to-the-moment, mission-critical information. Through the application of these techniques and technologies, the reach and value of existing data warehouse and BI systems can be increased by one or more orders of magnitude.

There are five components to proactive BI: real-time warehousing, automated anomaly and exception detection, proactive alerting with automatic recipient determination, seamless follow-through workflow, and automatic learning and refinement. As we will see, wireless technologies have a key role to play in increasing the value and efficiency of several of these steps.

Real-time Warehousing

The business value of real-time analytics and business activity monitoring (BAM) are becoming increasingly clear. Today, many enterprises either have built data warehouses that contain some amount of real-time information or have plans to enhance certain parts of the warehouse with real-time or near-real-time capabilities. In our previous article ("Agility Training," May 28, 2002), we described strategies to overcome technical hurdles in this race to enable the real-time enterprise.

This move to real time is one of the driving factors in the proactive BI revolution. In the world of weekly or monthly data loads, warehouses would be updated over the weekend and users would expect new reports at the beginning of the week, causing a Monday morning effect in data warehouse usage. When data loads began occurring each night, this turned into the "9 a.m. effect."

But when data is loaded continuously, how should users know when to run their reports to find something of interest? Clearly, they can't sit at their computers pressing the refresh key all day. Even forcing users to manually run or retrieve reports on a daily basis is an unnecessary burden. There must be a better way.

Automated Anomaly and Exception Detection

The more a BI report is requested on an intraday or daily basis, the more likely the report is not being used for traditional slice-and-dice online analytic processing (OLAP). Rather, the report is being used to detect anomalies — things that are in some way out of the ordinary — or gain a sense of comfort that there are no anomalies and that everything is operating as usual. Unfortunately, this method requires users to review long reports and visually pick out anything atypical or exceptional.

When warehouse data changes in real time (or even daily), a much better approach is for the BI system itself to monitor the data and the corresponding metrics, analytics, and reports, automatically detect conditions that are likely to interest various users, and proactively communicate these conditions to the appropriate user or group.

Automatically detecting exceptions and anomalies sounds complicated and conjures images of complex algorithms and neural network systems. Fortunately, anomaly detection doesn't need to be overly complicated, and there are simple approaches to automate the processes that users currently perform by hand. By calculating the usual value of a particular metric along various dimensions and at various levels of aggregation, a proactive BI system can then determine if the current, real-time value for that metric is either normal or in some way anomalous or exceptional.

For example, a proactive BI system may discover at a large retailer that sales of DVD players have suffered an unexpected drop in a certain region over the last few days. This insight can be achieved by comparing current period sales to last-year or last-week sales, or by comparing the sales trend for the product category across various regions and by finding one region that isn't behaving like the others.

The good news is that this type of calculation can be performed with standard BI tools using relatively simple calculations and basic statistics. But when a BI server has identified this potential problem, how does it get this potentially business-critical and time-sensitive information into the hands of people who can assess it and take corrective action?

Proactive Alerting and Automatic Recipient Determination

Most BI vendors currently approach this problem by using an information distribution server to send this information to recipients via email or wireless devices. In most cases, this approach requires a time-consuming process in which an administrator attempts to determine all the possible alert conditions and manually matches possible alerts to the appropriate recipients. The alternative — requiring users to use complex subscription systems to specify in detail exactly what type of information they want before receiving anything of value — is similarly ineffective. The result generally is low usage and low return on investment for the system.

A much better approach is to allow the system to automatically determine who should receive various types of alerts, much the way it can automatically look for anomalies and exceptions. Based on a structured view of an organization, such as a lightweight directory access protocol directory or a PeopleSoft or other HR system, it's possible to automatically determine the basic organizational structure of a company and the likely responsibilities of the employees.

In our DVD player example, the insight derived from these systems can allow our system to decide to send an alert about the low sales anomaly to the electronics category manager at headquarters and to the regional manager in charge of the stores where the problem is occurring.

Because these systems also contain contact information such as email address and wireless phone number, the alerts can be delivered to the appropriate employees without any prior setup, subscription, or configuration on the user's behalf. But what happens next?

Seamless Follow-through Workflow

Anomalies and exceptions detected by the data warehouse often indicate a condition that requires action. With this fact in mind, users should not only receive the alert, but should also have the ability to take immediate action. For example, the electronics category manager or the regional manager might decide to investigate whether or not there are DVD player inventory issues in that region that may be causing low sales volume — the alert recipients should be able to access this information with one click.

• Privacy
• Your California Privacy Rights
• Copyright © 2008 United Business Media LLC