Intelligent Enterprise

Better Insight for Business Decisions

RSS
Webcasts
Whitepapers
Subscribe
Home

August 10, 2003

Advanced Analytics Return to BI

BI tools are quietly gaining statistical depth, but exploiting the new possibilities isn't straightforward

by Seth Grimes

Ease of use and accessibility are essential to widespread acceptance of BI, but simplicity has unfortunately come at the cost of analytic depth. In "BPM Drumbeat" (April 22, 2003), I mentioned that BI's online analytic processing (OLAP) focus has meant neglect of valuable approaches to problems that involve nonlinearity, uncertainty, indeterminacy, and time complexities. I asked readers to email me for a more extensive explanation. Enough readers took me up on my offer that I turned my responses into this column.

Past, Present, and Future

Nigel Pendse, lead author of the OLAP Report, recently wrote to me, "Without any scientific basis, I'd say that use of stats in business analysis has been declining over the years. Today's mainstream BI products typically offer less stats functionality than their 1970s predecessors, presumably because few people used [the statistical capabilities]."

Indeed, the analytic capabilities of mainstream BI tools haven't, until recently, gone far beyond aggregation (roll-up) and computation of derived values using basic algebraic formulas. The largest part of BI's value stems from reporting, query, and visualization interfaces rather than from analytic sophistication. While there's not necessarily anything wrong with stressing interfaces for exploratory analysis, activity and performance monitoring over analytic depth, users often think that dashboard displays and OLAP-style slice and dice are adequate when they only scratch the surface.

In my April column, I wrote about BI vendors' new BPM offerings for the reintroduction of sophisticated statistics into mainstream business analytics. BPM methodologies rely heavily on advanced statistics. If software tools are to successfully encapsulate Six Sigma and similar approaches and provide sophisticated forecasting and optimization capabilities, they'll need to bring advanced stats into the mainstream.

Pendse wrote me further, "In my own experience, apps that used advanced stats and predictive models have been less successful, in business terms, than simpler ones." Perhaps. Nonetheless, mainstream BI tool vendors have quietly beefed up stats capabilities, responding to external pressures — particularly the inclusion of aggregation and descriptive-statistics functions in the SQL-99 query-language standard for relational databases, as well as the BPM push. In some cases, the roots of the change have been almost accidental. Statistics powerhouses SAS and SPSS have been paying increasing attention to the BI market, and Informatica and a few other vendors have moved to a Java 2 Enterprise Edition (J2EE) architecture, which allows external statistics routines to be invoked.

Whatever the reasons, enterprises should consider exploiting the newly available advanced functions. Usability, starting with users understanding feature benefits, is a large obstacle. This is where, in the words of Business Objects product strategy VP Alex Moissis, "BI tools have been 'stuck in the middle' between trying to deliver powerful algorithms while also often trying to present the utilities in an environment that is easy to understand and use." Moissis thinks the opportunity is in ensuring that "application designers will have the flexibility to capture powerful analytics when defining metrics to be viewed by nonexpert end users."

Fred Richards, marketing systems director at MicroStrategy, envisions the same scenario where "typically the statistical functions are embedded in a metric that appears on a report. A Ph.D. may develop the right function, decide upon the sample size, etc., and then build this function."

While vendors eventually might find a way to embed sophisticated statistics into routine applications, in the near term, the key to exploiting advanced analytics will necessarily be business-analyst awareness of the limitations of current methods.

Neglected Approaches

I'll elaborate on the four neglected problem characteristics that I listed in my April column and cite cases where solutions are already found in shipping BI products.