Advanced Analytics Return to BIBI tools are quietly gaining statistical depth, but exploiting the new possibilities isn't straightforwardby Seth Grimes Continued from Page 1 Nonlinear problems are best modeled by other than a straight line. Take anything that's cyclical, such as business cycles or the climate. Even in the tropics, the temperature is cooler over night and warmer during the day, cooler in the winter and warmer in the summer. The effect is a composite of cycles with daily and yearly frequencies. Cycles of this nature are best modeled with sinusoidal (trigonometric) functions. Sinusoidal functions are nonlinear. If you model temperatures during the day with a straight line using the linear regression you get with, say, Microsoft Analysis Services the daily cycle will be flattened out. The model will have no predictive power, although it may indicate a trend toward cooler or warmer weather depending on the season. Follow that trend and you will believe your community will either freeze over, with temperatures eventually dropping below absolute zero, or it will vaporize in temperatures that exceed the sun's. One answer to this particular linearization deficiency is called logistic regression. You model linearly the logarithm of the "odds ratio," which has the effect of forcing values to stay within defined bounds rather than trending off to infinity, but still gives you the simplicity of a linear model. Although I don't know of any mainstream BI tool that offers that option, several do at least provide a more extensive set of options for handling nonlinear problems. One example is Hyperion's Essbase XTD, with strong roots in financial analysis. For a problem that includes uncertainty, which is typically expressed with a probability density function, only the broad characteristics of the outcome are predictable. Take, for example, queues with random arrival times — such as at tollgates or bank-teller windows — which are typically modeled with a Poisson distribution. Without the probabilistic term, you might be able to accurately guess an aggregate number of arrivals in an extended period. But you'd be hampered in determining, say, the number of tollbooths or checkout counters you'd need open to handle peak hours. A problem that includes uncertainty is nondeterministic; that is, the results of particular circumstances can't be known. The difference between indeterminacy and uncertainty is small. Rather than giving a lecture on almost nothing, I'll take a shot by saying that that an indeterminate event is one whose outcome may or may not fall into a defined set of possibilities but definitely can't be known beforehand. Events such as flipping a coin or opening the box that holds Schrodinger's Cat are examples. Flip a coin a thousand times, and you'll get an idea of the probability of a heads, tails, or edge landing. What is indeterminate in the single case becomes predictable in the mass of cases. I was surprised to find that MicroStrategy now supports a broad range of probability distributions usable in creating metrics, but there you are. BI consultant Neil Raden put it well in a discussion when he said that "the purpose of statistics in a tool like MicroStrategy will be for well-thought-out, persistent statistical models that drive things like alerts, forecasts, and dynamic models, such as yield management, dynamic pricing, and, especially, repackaged verticals. These will not be stream-of-consciousness investigations; they will use the application of carefully designed and tested, fixed, statistical models." Last, I've been known to harp on BI-tool shortcomings in handling time complexity. Just about all BI tools now deal with basic calendar frequencies: days, weeks, months, quarters, and sometimes subday periods and irregular time points. And most of them let you convert series formed from sequences of observations over time of a given value from one frequency to another, by a variety of roll-up and disaggregation (allocation) methods. I have yet to see one that offers significant ability to model, analyze, and forecast time series. But that will no longer be true if SAS BI interfaces due in early 2004 deliver on their promise of providing access to the full range of SAS capabilities, including the Econometric and Time Series (ETS) module. However, these functions will remain far from common unless users push other vendors to provide them. Demand ResponseWhatever the reasons — market forces, the growth of BPM, or the emergence of standards-based architectures — BI users have a new opportunity to improve the accuracy of their analytic systems. Poor usability remains an obstacle to the adoption of advanced analytics. But vendors that have taken steps to augment the sophistication of their offerings will probably be willing to work to overcome problems if they see a demand. Seth Grimes [grimes@altaplana.com] is president and principal consultant of Alta Plana Corp., a Washington, D.C.-based company specializing in analytic computing systems and demographic and economic statistics. RESOURCESRelated Article at IntelligentEnterprise.com: "The BPM Drumbeat" April 22, 2003
|
 |
Most Popular This Week
IE Weekly Newsletter
Subscribe to the newsletter
| |
| |||||||||||||||||||||||||||||||
