Intelligent Enterprise

I was more than a little surprised when I read the article "Think Critically When Applying Best Practices," by Bob Becker and Ralph Kimball. Unless I misread it, they have come around and defined BI as the total process, including data warehousing. This is something that the other prominent data warehousing guru's did a few years ago when, fearing they would miss the boat of the suddenly hot BI market, declared their IT-oriented data warehouse environment as BI. The fallacy in this is that the people who use BI were always conspicuously absent from the diagrams and descriptions of the data warehouse. Their architecture blueprints depicted "users" (and keep in mind that there are only two industries that call their customers users) as little stick figures crushed under the weight of their elegant, multi-colored architectures, or through demeaning models with names such as "Farmers."

I have always identified myself, as far as data warehouse design and methodology go, as firmly in the Kimball camp, going back more than a dozen years. This whole industry owes Ralph a deep debt of gratitude for meticulously informing and explaining how to build successful data warehouses. Until I stopped doing that myself a few years ago, I never missed an opportunity to attend to a Kimball class or to read his books and articles because I always came away with something I didn't know, complete with a roadmap for doing it. Part of the reason behind this was that Ralph and his team never stopped working at it, delivering actual data warehouse designs to actual customers.

In contrast, the other prominent guru's of data warehousing rarely if ever got their hands dirty. Their high-level architectures and methodologies led many down the primrose path to over-designed, latency-laden, impossible to build and maintain structures that served no one except those who profited from the effort. There was no understanding of how, or if, the data warehouse and BI could be useful.

Consider this - if data were smart enough, endowed with the contextual information to be self-describing and unambiguously accurate semantically, and there were sufficient bandwidth and processing power to query original data for BI without extracting it and maintaining another repository of this transformed data, there would still be a need for the consumers of this information to understand it, manipulate it, share it, model with it and, for that matter, for unattended agents to do the same. That is BI. So as far as I'm concerned, the data warehouse and its processes are not BI at all, they are an IT-centric monolith that enables BI. It's the plumbing. Someone who is an expert at this is not necessarily an expert at BI. In fact, in my experience, the two are almost mutually exclusive. Data warehousing, either Kimball fashion or otherwise, is about as far from BI as Canarsie is from Times Square.

When Howard Dresner coined the term in 1989 (and he was still at Digital, he didn't join Gartner for two more years), he was referring to the software tools that provided reporting, analysis and planning capabilities to business, not data warehousing. Data warehousing had not quite emerged at that point.

If you search the literature, you will find that BI and DW were on separate tracks. Lumping the IT-centric DW process in with BI is very misleading. BI is directed at knowledge workers and those they inform, and, only recently, lower-level operational decision-making and more automated processes such as rules engines or what Fair Isaac calls Enterprise Decision Management (full disclosure: I just wrote a book with James Taylor of Fair Isaac). Data Warehousing is a cumbersome, lengthy process of gathering second-hand data and creating a permanent repository.

It is fashionable in DW these days to speak of the "Single Version of the Truth," but in the BI world, it is more important to consider different possibilities and scenarios. This is a stark delineation between the two and explains the typical chasm between the IT organization that manages the DW and the business users who work with BI. So my vote is to strip data warehousing of its BI credentials because it only serves to further confuse people. And frankly, BI is going to be with us for a long time, but data warehousing has pretty much run its course.

Neil Raden is the founder of Hired Brains, providers of consulting, research and analysis in Business Intelligence, Performance Management, real-time analytics and information/semantic integration. Don't miss Neil's many insightful articles in the Intelligent Enterprise archive.