Analytics on Demand: The Zero Latency Enterprise

The ability to collect and analyze information in real time is a cornerstone of the "intelligent" organization

One vendor that is placing significant emphasis on realtime data warehousing and decision processing is Compaq Computer Corp. with its ZLE solution. ZLE consists of a technology architecture, Compaq hardware and software, consulting services, and third-party products. (The ZLE concept originally surfaced in a research paper by Gartner Group, and is now a part of Gartner's Business Activity Monitoring initiative. ZLE also borrows concepts from the book The Power of Now by Vivek Ranadive, the founder and CEO of Tibco Software.)

One of the early adaptors of ZLE is Target Corp., which is using the technology in a CRM initiative that involves a multiterabyte, realtime data warehouse containing details about customer interactions across its 900 stores, Web sites, call centers, and catalog sales.

Realtime Analysis Engine

To provide fast access to information, data warehousing applications precompute frequently requested data summaries and key business performance metrics. If these summaries and metrics satisfy business users' needs, then it is a simple matter to provide rapid and easy access to this information via Web-based technologies such as a corporate portal.

For a realtime data warehouse, an event-driven analysis engine employing underlying data summarization and OLAP tools can be used. These tools asynchronously create data summaries and business performance metrics on a regularly scheduled basis, or based on business rules that are applied to incoming information as it is added to the data warehouse. The summaries and metrics can be stored in the data warehouse and delivered as required to business users via a corporate portal.

Fast access to information is inhibited when data summaries and business performance metrics don't meet business or application needs, and the analysis engine must create this information synchronously in real time. Whereas vendors are constantly improving the performance of their decision-support systems using new data management techniques, parallel processing, and so on, there is no simple performance solution when large amounts of data have to be analyzed to calculate a single performance metric. Certain types of decision processing, however, lend themselves to a predictive approach that can employ the services of a data-mining-driven decision engine, and thus avoid the need to analyze data warehouse data in real time.

Realtime Decision Engine

Traditional decision processing involves users applying their business knowledge and expertise to information coming out of a data warehouse. For many business situations, this manual approach to decision-making is simply too slow.

If, for example, a consumer visits a bank or a bank's Web site and applies for a loan, it is becoming increasingly important for the bank, from a competitive standpoint, to be able grant (or reject) the loan while the person is in the bank or connected to the Web site. This decision may involve analyzing the consumer's current business with the bank, assessing the creditworthiness of the client, determining risk, and so on. Clearly, the decision-making process must be automated if a loan decision is to be made in real time. This goal is achieved using the services of a decision engine.

Decision engines can exist as standalone solutions, but frequently they are embedded in other software such as Web application servers, corporate portals, or analytic applications. The basic architecture of such products, however, is the same. The two main inputs to a decision engine are a set of business rules and the set of information that the business rules are applied to in order to make a decision. The business rules encapsulate knowledge about specific business situations (how to assess the risk of making a loan, for example). These rules may be created by business users, or may be generated automatically by tools that observe, analyze, and learn about business processes. Examples of products that generate business rules automatically include data mining tools and collaborative filtering products that monitor the behavior of Web site users.

The set of data that the business rules are to be applied to may come from a data warehouse application, or may be contained in a realtime request from a business user or e-business application. Returning to our bank loan example, the data entered in a realtime request by a loan manager could consist of the client's salary, age, profession, loan amount, and so on. The decision engine can then apply the business rules to this set of data to determine the risk associated with granting the client a loan. If the risk is low, the decision engine could recommend to the loan manager to grant the loan. If the loan request is coming from an Internet user, the decision engine may make the decision to grant the loan automatically.

There are many different types of decision engines. Here, I will briefly review product examples of data mining tools, CRM analytic applications, and Web realtime personalization servers that can be used for automated decision making.

In order to make data mining technology more usable, data mining software vendors have been separating their products into a development environment and a deployment environment. The development environment is used by experts to build and test a data mining model against existing operational or data warehouse data. When the model (in essence a set of business rules) is finalized, it is copied to a separate deployment environment where it can be easily invoked and a set of data values passed to it for evaluation in real time.

IBM's DB2 Intelligent Miner Scoring, for example, enables an application to invoke, using a simple SQL statement, a data mining model stored in a DB2 database. This facility is used to score (segment, classify, or rank) records based on a set of predetermined criteria expressed in the data mining model. SAS Institute Inc.'s Enterprise Miner product can generate data mining deployment models in SAS, C, and Java code for inclusion in an external decision engine. A data mining deployment model acts as a decision engine that uses predictive techniques to make a decision or recommendation. The predictive model is a set of business rules that has been created by analysing existing business operations using information from an operational system or data warehouse. This approach is particularly useful in a CRM environment for doing one-to-one marketing.

Rate This Article Rating: 1 (best) 2 3 4 5 (worst) Comments: Optional e-mail address:

Several vendors are also offering CRM analytic applications with embedded decision engines that use business metrics generated from a data warehouse to drive front-office operations in support of an automated closed-loop decision-making system. Examples of products here include BusinessObjects Application Foundation, E.piphany Real-Time Personalization, and Xchange Real Time.

Another type of decision engine can be found in the personalization facilities of Web application servers. These personalization capabilities use business rules to determine in real time the Web pages viewed by e-business users such as consumers on the public Internet. These rules-driven tools can often also be used in independent decision engines. IBM's WebSphere Web application server, for example, offers two different personalization tools: Macromedia Inc.'s LikeMinds, which makes recommendations based on historical behavior through collaborative filtering; and Brokat's Blaze Advisor, which is used in Compaq's ZLE solution discussed earlier. With this product, the text-based business rules can be defined by the user or generated using the output from a decision-support system. SAS, for example, can connect the deployment environment of its Enterprise Miner product to the Brokat Blaze Advisor.

The Realtime Future

You can see from this discussion that a realtime decision-support system provides realtime data warehousing, realtime information analysis, and realtime and automated business decision-making. These components may be used in conjunction with each other or independently.

Realtime decision processing is poised to become one of the key driving forces behind the next generation of decision-support systems and will be part of almost every intelligent business over the next few years.

Colin White [cwhite@databaseassociates.com] is the president of DataBase Associates International. He specializes in data warehousing, business intelligence, and corporate portals.

RESOURCES

Compaq ZLE: www.compaq.com/zle

Informatica PowerCenter: www.informatica.com/products/infrastructure/pcenter.htm

Ranadive, Vivek. The Power of Now: How Winning Companies Sense and Respond to Change Using Real-Time Technology. McGraw-Hill, September 1999

Related Articles at IntelligentEnterprise.com:

"The Living Transaction," May 24, 2001: www.intelligententerprise.com/010524/feat3_1.jhtml

"In Living Color," Aug. 18, 2000: www.intelligententerprise.com/000818/feat2.jhtml