The Missing Link

Will faster OLAP presage an evolutionary advance - and more time on the golf course?

Stormy, primordial weather in Palm Springs: I couldn't help but feel sorry for the business-casual travelers lugging their golf clubs away from the baggage claim area at the airport. Instead of desert days of sun and double birdies, they would have to endure streams in the streets and 40-degree nights. Nor could the writer in me resist reaching for the easy metaphor: storms in paradise, storms over the economy, gratification delayed, if not denied. That Calloway Big Bertha might never touch the tee.

We all know the story: tumbling stock markets, collapsing consumer confidence, and a new economy veering rapidly into bankruptcy. From Alan Greenspan's monetary micromanagement to mad cow disease, we could assign blame to a number of causes, but the important question is: Who will survive in the aftermath? Which existing organizations will evolve and adapt in ways that enable them to tap the power of information and compete with future upstarts - aggressive new ventures that will one day define the next "new" economy?

These thoughts framed my visit to The Data Warehouse Institute's (TDWI's) recent conference in Palm Springs, Calif. the week of February 26. While other conference/trade show producers left the data warehousing arena to chase the buzz of hotter fields such as CRM and e-commerce, TDWI has prospered by sticking close to its core competency: that is, training-style sessions by leading consultants in data warehousing. Certainly, where data warehousing has become the centerpiece of analytical activity for e-CRM and e-business applications, TDWI instructors have addressed the business context: but for the most part, people come to TDWI to attend a boot camp of sorts on how to build and manage data warehouses and data marts.

In Palm Springs, TDWI stepped out a tad by offering a special summit, "The Next Frontier in Business Intelligence: Advanced Decision Support Techniques" led by Erik Thomsen of Dimensional Systems. (Erik is also a contributing editor to this publication.) Scheduled to the gills with meetings, I wasn't able to attend as much of this special summit as I would have liked, but the exposure sharpened my perspective on the state of the art in building analytic solutions. As organizations fight their way out of the current slump, they will be relying on BI specialists to give decision makers a timely and complete view of information; then, decision makers will have the intelligence necessary to point their organizations toward higher-value business and industrial processes.

Down the road, when economic clouds dissipate and the next growth phase gathers momentum, economists, much like anthropologists studying apes and chimpanzees, will sift through our current tribulations for evidence of the "missing link" that connects the old with the new. They might even point to one organization that was able to put it all together - to assemble the knowledge, organization, and active processes that enabled it to step ahead of the pack and into the next "new" economy. My guess is that the missing link will lie in how that organization broke out of an earlier, static age of BI and data warehousing toward smarter, more adaptable uses of analytic solutions. Our anthropological economists will be looking for the first intelligent enterprise.

BI: Exploding the Cube

If there was an 800-pound gorilla on TDWI's show floor, it was the "T3 Business Intelligence Project," a joint demonstration by Microsoft, Unisys Corp., Knosys Inc., and EMC Corp. The first thing to note was Unisys' innovative Intel processor-based ES7000 server, which features the company's Cellular Multiprocessing architecture: 64 gigabytes of shared memory and a wide bandwidth "crossbar" technology to interconnect up to 32 processors for high-volume I/O activity. The ES7000 has become a critical demonstration platform for Microsoft's Windows 2000 Datacenter Server, SQL Server, and Analytical (OLAP) Services.

The T3 configuration showed an eight- processor data warehouse server and 16 processors devoted to the OLAP cube, which Microsoft said was built with 1.2 terabytes of source data. The demonstration involved complex queries, which brought the EMC storage area network (SAN) and Connectrix fibre channel switch into the picture. Knosys, developer of the ProClarity component-based analytical front-end environment and a close partner with Microsoft, provided a custom sales analysis interface. The T3 prototype was designed to analyze 80 markets, far more than any current solutions, according to Microsoft.

Keeping in mind that this was only a demonstration - in other words, don't try this at home, at least not just yet - the intent was to blow people away with the capacity not only of each element of the T3 configuration, but also the system taken as a whole. Microsoft, of course, was hoping attendees would take home an unforgettable perception that its stuff has what it takes for enterprise-level activity; in other words, move over, IBM, Oracle, and NCR/Teradata.

At this point, judging from my anecdotal conversations with attendees, Microsoft still has a ways to go before it cements such an impression. According to TDWI's Industry Study 2000 survey, Microsoft's database and analytical platforms trail the leaders for CRM projects, undoubtedly the hot spot for current BI activity. However, as Microsoft solidifies important (and, of course, favorable) standards for OLAP query and data integration and proves its mettle with critical IT decision makers, the numbers could change.

MOLAP Momentum

Perhaps an even bigger impact of T3 was the excitement it generated regarding the potential of multidimensional OLAP (MOLAP) databases. The once-great MOLAP vs. relational OLAP (ROLAP) debates died out a few years ago, with ROLAP winning on basic performance, scalability, and manageability. Also muting the debate was that most OLAP and RDBMS systems found ways to do both, depending on the requirements and configuration - but leaned heavily on the relational engines for all aspects of scalability.

Internet and caching architectures, however, could propel MOLAP aficionados - not to mention business analysts crying out for more sophisticated, realtime analysis - into higher states of ecstasy than writing complex SQL queries could ever bring. With demands for performance and greater scope of analysis creeping ever higher, new solutions are arriving that will give BI professionals reason to take another look.

I met with a couple of vendors sporting impressive tools that could change the performance dynamics of multidimensional OLAP. The first was QueryObject Systems Corp., which offers a hybrid solution that can support MOLAP cube and relational table views simultaneously within the same data mart. Using precalculated aggregations and a polynomial index structure, QueryObject claims it can support much higher numbers of dimensions and query speeds than other hybrid solutions that lean heavily on standard relational indexing and optimization. The company is interested in making customers out of organizations that have nonrelational legacy systems - such as Mumps, which is still the dominant data store in the health care industry.

QueryObject is also Internet-savvy: Borrowing a page from Akamai Technologies Inc., the company's Distributed Replication Server can "stream" analytical data on demand, to be cached locally for more efficient performance. In other words, rather than Web-enable the tools, QueryObject aims to Web-enable the data. "Time to first query is the name of the game," said Matthew Doering, the company's senior VP and CTO.

The second vendor to make an impression was HyperRoll Inc., led by CEO Yossi Caspi, who has a long history in data warehouse solutions development in Israel and was once senior product manager with Oracle's OLAP product division. CTO Reuven Bakalash and COO Arik Paran also have deep experience in Israel's enterprise software industry. Like QueryObject, HyperRoll is aiming at high-performance data aggregation and calculation; however, the company positions itself differently. HyperRoll views itself as a "plug-in" to existing systems, whether relational or not. "We're the backhoe coming to the aid of someone using a shovel," Paran noted.

With e-business and e-CRM creating hot demand for a faster closed loop between operational and analytical systems, HyperRoll is focused on accelerating BI through greater flexibility in the number of dimensions supported and dramatically improving the overall level of the analytical systems' responsiveness. As a simple plug-in caching engine that can take advantage of fast access storage, HyperRoll positions the product as a performance accelerator for ROLAP, MOLAP, and hybrid systems that does not itself create a management headache for DBAs and data warehouse managers.

My notes are brimming with other observations from the conference, which I'll have to save for a later column. And once the weather clears, I'll continue my own search for the missing link: both on the golf course and inside the next generation of multidimensional OLAP architectures.