Intelligent Enterprise

At the conclusion of Business Object's Influencer Summit yesterday, Jonathan Becher hosted a wrap-up Q&A with Doug Merritt, Marge Breya and Sanjay Poonen. Rather than attributing quotes to each executive, I've consolidated the responses on five topics:

"It's a case where one plus one equals three." Speaking at the first-ever Business Objects Influencer Summit in Boston this week, this is how Sanjay Poonen, SVP and GM of Performance Optimization Applications, explained an increase in BI revenues at the company since it became a unit of SAP.

Normally, following an acquisition, sales decline for the first year or so. Not so with SAP's acquisition of Business Objects, with Poonen claiming sales were 30% higher in the first half of the year compared to 2007. He explained that there is a difference in market dynamics when a market leader acquires another leader versus a niche player. Surprising as well is that company officials estimate half the sales came from new accounts, so the strong performance is not only from Business Objects tapping existing SAP customers.

I was in rainy Boston yesterday at the Business Objects Influencer Summit, which was kicked off with Jonathan Becher, SVP of Marketing for Business Objects. It's a very process-oriented message (which explains why I'm here): using business intelligence to drive process efficiency, improve insight to close the gap between strategy and execution, and add flexibility to create new business processes that align operations to strategy.

Becher was joined by Doug Merritt, EVP and GM of Business User Global Sales (moving from a product role), who continued with the message of how total insight allows organizations to optimize business performance. Merritt discussed a number of customer case studies, focusing on how their easy-to-use end-user tools are being used to solve real business problems.

With apologies to Gertrude Stein, there's not enough "there" there in the business rules management system market, what with only a handful of players, but yesterday's announcement by IBM that it will acquire ILog will certainly spark aftershocks. I came across a few particularly keen questions from a former industry insider.

To go straight to the source, I first spoke to an ILog exec yesterday who shared this bottom-line assessment of why the timing for this deal: "The market is maturing, and business rules are taking a legitimate position in infrastructure," said Jean-Franηois Abramatic, Chief Product Officer. "It's clear now that business rules are an essential part of business process management/services-oriented architecture platform."

IBM today announced that it plans to acquire ILog, an unquestionable leader in the business rules engine marketplace. The acquisition comes at a time when ILog seemed to be faltering, with declining profitability and reliance on a troubled financial sector, but there's no doubting the tremendous value to IBM and customers.

IBM is not new to business rules engines (BRE). WebSphere has a rules component, and IBM has experience with various other rules integration models (e.g. PegaSystems, Haley etc.) as well as with in-house experimentation. Yet, IBM has always lagged in its BRE capabilities. In contrast, ILog is a known market leader with formidable capabilities and established market presence – Forrester ranks ILOG and Fair Isaac as the top two BRE vendors. Pegasystems and Corticon are the next largest competitors, while Haley was recently acquired by Australian company RuleBurst.

I was going to just link to Mike Kavis' post on the Top 10 Reasons Why People Are Making SOA Fail, but I wanted to added some of my own comments. By the way, he's talking primarily about IT people, not business people, in the fail part of the equation.

Number 1 reason: they fail to explain SOA's business value. Kavis recommends (and I completely agree) starting with business problems first, specifically using BPM as the "killer app" to justify the existence of SOA.

That's right. We read this all the time: In order to succeed, CIO's should be business leaders. But the fact is, leading the business is not the CIO's business. Yet that's not bad news... in fact, it actually makes the CIO more influential.

The fact is, heads of Marketing, Operations, Procurement etc. lead the business. The head of IT does not, by definition, lead the business. And this is true even the business is information or technology. For example, the CIOs of CA, Forrester, Gartner, Google and Microsoft — to name just a few information/technology providers — do not lead their businesses; their counterparts in sales & marketing and operations do. (Interestingly, the CTO for an information technology firm is much closer to being a business leader, because the CTO owns or advises the product strategy.)

Gary Cokins of SAS and James Taylor, my partner at Smart (enough) Systems, in an admirable attempt to disambiguate the terms Enterprise Decision Management (EDM) and Performance Management have, unfortunately, both gotten it wrong.

Gary claims that James "marginalizes Performance Management as being too narrow." Instead, he (Gary) suggests that "Performance Management and EDM are arguably very similar." James claims that EDM "goes one step further."

Are you as sick as I am of so-called "architects" swiftboating BPM with phony strawman arguments? Here's the latest, from blogger Nick Malik:

I like point out really nutty ideas, even when a lot of people have spent a lot of time investing in them... [BPM] created pretty languages for describing business processes, and we started telling the business that once business processes are described using these languages, then you can push a button and "viola" the process becomes automated. According to the 'true believers,' we can give end users one of our pretty languages (BPMN or BPEL) and they will write their own software, and we can fire all the IT developers.

Over at the Adobe Developer Connection Web site, Belgian developer Sιbastien Arbogast has posted an interesting article (a tutorial of sorts) on how to write next-generation Web apps in Flex. What's interesting isn't the Flex part (or the demo app itself, which is rather uninspired) but the underlying stack, which gives some hint, I think, of what Adobe's Flex evangelistas may be envisioning as LAMP-Next. It's a combination of Flex (for the presentation layer), BlazeDS (for messaging and presence), Spring (the runtime framework), Hibernate (for persistence), and MySQL (data layer). The application server used in Arbogast's example happens to be JBoss, but it could just as easily be something else.

Yesterday, Oracle lifted the veil on its plans for BEA. Naturally, Oracle said the acquisition as a whole was not just for market share, but for BEA's technology, which would all become part of the Fusion middleware platform. There was a lot of material presented, but I'll focus on the product convergence plan as it relates to business process management suites (BPMS).

To rationalize the product set, Oracle first sorted the BEA product catalog into one of three buckets: 1) strategic, where BEA was considered superior to existing Fusion components or a new capability; 2) continue and converge, where BEA component would be positioned as secondary, maintained but eventually merged into the current Fusion offering; and 3) maintenance, mostly OEM offerings, which it seems Oracle wants to walk away from as soon as they can. The BEA installed base was reassured that all BEA current products would continue to be "supported," although those that are not "strategic" would not be enhanced.

There's something vaguely Orwellian, at times, about the language that turns up in quarterly and annual reports (the kind U.S. public corporations are required to file with the Security and Exchange Commission). Remember the classic slogans from Orwell's 1984? War is peace. Freedom is slavery. Ignorance is strength.

Perhaps we should now add, "Higher prices mean lower cost of ownership."

I'm reading a well-known software company's quarterly report dated April 1, 2008, wherein the following rather noble-sounding statements are made:

Fujitsu is releasing version 10 of its Interstage BPM, and I had a chance for an in-depth demo a few weeks ago in advance of the recent announcement. On the design side, their new version of Studio now allows business analysts and IT to work together, and it includes forms development. In terms of end-user functionality, there have been improvements to workflow to enable collaboration and new dashboard functionality. Most exciting, I think, is full support for multi-tenanting to allow for shared services and SaaS.

The most interesting keynote at last week's Intalio User Conference was by Greg Olson, founder of Coghead, a BPM-in-the-cloud service that uses Intalio as the process engine under the covers. Coghead bills itself as a next-generation platform for situational apps, such as built today on Excel, Access, or FileMaker. Instead of professional developers, Coghead targets independent Web developers and power users. The platform is 100 percent Web based, a multi-tenant service hosted on the Amazon cloud infrastructure, with simple subscription-based pricing (free for single user). You can define data, forms, and perform the usual set of database operations, so it's really easy to build a database app in the cloud.

Homeowners know that installing energy-efficient windows helps save money in the long run, yet many are reluctant to make the investment in these challenging times. Businesses are no different, but even in this difficult economy, companies looking to optimize business processes have a very useful yet inexpensive tool at hand. It's called the Hawthorne Effect...

E-discovery and auditing are flip sides of a single coin, the one concerned with retention of records and their production in litigation, the other with studying records to verify the correct of execution of corporate business processes and accounting procedures. Extending the metaphor, compliance is the coin standing on edge: neither anticipation and response to litigation (e-discovery) nor historical analysis (auditing) but rather operational rules and monitoring designed to ensure that businesses stay out of legal and accounting trouble.

Did you know Adobe had a business process management suite (BPMS)? Most people don't, even though with more than 5,000 customers they could be considered a major player. One reason people don't know about Adobe and BPM is that the company doesn't talk about it in the usual way. In fact, it treats the normal catalog of BPMS features and functions, like workflow and integration adapters, as commodities. For example, Adobe includes process modeling and a workflow engine inside every copy of LiveCycle Enterprise Suite, although to get full human task support you need to get the Process Management ES component as well.

It seems obvious that enterprise IT shops take Rich Internet Applications and Web 2.0 with a large grain of salt. It isn't that managers don't listen to enthusiastic developers or ignore the industry hype. However, most of them are in the business of insuring the cart stays behind the horse – the horse being the enterprise's existing servers, data systems, and applications. Most of the big players in Web 2.0 for the enterprise get this. They carefully position their enterprise RIA/Web 2.0 technologies and products near the margin – an add-on, a pilot for a new direction, nothing too radical. That's essentially what IBM is doing with mashups, Microsoft is doing with Silverlight, and Adobe is doing with its newest release of LiveCycle ES – Update 1.

Almost two years ago, I reported on the IBM acquisition of FileNet, wherein I quoted their plan to "integrate IBM's BPM and SOA technologies with the FileNet platform." I interpreted this to mean that FileNet BPM could finally get separated from its document-centric chains, and become the product that it should have been years ago. Just as Jessica Rabbit said "I'm not bad, I'm just drawn that way," the FileNet BPM wasn't (isn't) document-centric, it's just marketed that way.

As the former director of e-business evangelism for FileNet in 2000-1 when they were launching this generation of the BPM product, I had some idea of what I was talking about — I saw that 40% of the BPM installations in some countries did not involve documents at all, and that this was due to the local sales and marketing messages and techniques rather than any inherently different BPM requirements between countries. So several years after I left FileNet, when the acquisition occurred and I saw that initial press release, I imagined that the best possible thing would be if the BPM product were to be separated out and made part of the IBM WebSphere suite, in order to flesh out the badly-needed human-facing workflow side of things over there. I realized that would mean some major surgery on the product, but a stronger unified BPM suite would emerge from that.

I see my friend Jesper is moving on from BEA, so the reality of the Oracle acquisition is finally sinking in. When I hear people say that Oracle bought BEA for their BPM, I have to laugh. I'm fairly confident the Oracle crew that went after BEA could not even spell BPM. But no doubt the two BPMSs will have to be merged or rationalized somehow into a single primary offering (although IBM/FileNet provides an example of how that can be dragged out for years). I don't actually know what Oracle's plans are, and they haven't solicited my opinion — you can be sure that if they had, I would not be writing about it. But I have thought a bit about what they ought to do.

Lombardi's Jim Rudden posts an admittedly "cranky" piece about software giants like SAP crashing the business process management (BPM) party. His beef with those companies, which he calls "Stackers," is that they pursue the promise of BPM half-heartedly. Actually, they have done everything in their power to bury BPM deep in what they view as their real market… which in the case of SAP and Oracle, he says, is enterprise apps, and in the case of IBM is… well he's not sure ( I would say IBM Global Business Services billable hours). However, if those guys — none of whom can touch Lombardi for speed of development (agility!), business empowerment, and overall elegance in execution — were not succeeding at some level, Jim would surely not be so cranky. But I think he paints the Stackers in BPM with too broad and too black a brush. So let me offer a more nuanced view.

Regarding the BPMS Watch Ratings report I released last month, each of the 11 BPM Suites evaluated was scored on the same set of capability categories, based on a weighted list of features/attributes, including "Strength of Execution," representing a subjective catch-all attribute (the individual reports on each vendor are available here at BPMInstitite.org. Three process types described in the report — production workflow, case management, and integration-centric — apply different weightings to the various capability categories, but use the same score for each category.

I have been looking for a way to publish the details of the scoring, and at the same time allow users to apply their own weightings to the features in each capability category, as well as to create custom process types with their own capability category weightings. I wanted to do it online, not as an Excel download, but had no idea how to do that.

As an "Enterprise-focused" content management analyst, I am asked two basic questions on a regular basis. The first is "what about SharePoint?" The second is, "what about imaging?"

At many conferences, and regularly via e-mail, people ask me about imaging in the context of ECM. Imaging is the major cost that most projects either forget about or dramatically under budget for. During the buying process it's all too easy to get caught up in the flurry of believing that every file will soon be digital, even though paper is clearly here to stay.

That might be one way to restate the premise of a survey on the Business Process Management Notation standard that has stirred up quite a controversy. The survey is interesting (because it raises some some good questions about BPMN and business process modeling) and entertaining (because it challenges dogmatic thinking on the topic).

In nutshell, the researchers reviewed 126 BPMN diagrams collected from "consultants, seminar participants, and online sources" (in other words, more or less unscientifically, which of course does not automatically invalidate the research), and found that of the 52 distinct elements (symbols) that exist in BPMN 1.1 specifications:

- Only nine elements were used on the average in each diagram (i.e. less than 20%)
- Only five elements were used in more than half the models, and another six symbols in a fourth of the models
- 17 elements (more than 30%) were used in three or fewer models, including five elements not used at all!

I'm picking and choosing my sessions here at SAPPHIRE carefully, in part because I have some prearranged meetings specifically about BPM. I had a chance one-on-one meeting with Wolfgang Hilpert, SVP of NetWeaver BPM, this afternoon; funnily enough, just after I attended Ginger Gatling's session this morning, I had lunch in the press area, and when I mentioned that I'd seen the session on the new SAP BPM, three pairs of ears at the table swiveled around. These three, who I didn't know (nametags, unfortunately, hang below the level of the table when seated), gave me a light grilling on my opinions of what I had seen; although I figured that they worked for SAP, it wasn't until they stood up that I saw Hilpert's name tag.

By the time that we had our prearranged meeting, then, he knew that I'd seen a product overview, and he'd already heard my views on it, so we could jump right to some of the good stuff.

Surprisingly little information has reached public view concerning BPMN 2.0, the update of the Business Process Management Notation standard now under consideration in OMG. Unlike most standards approval processes, the outcome of this one is not preordained. There are two submissions, quite different, and it could go either way.

Oracle's Vishal Saxena notes that one reason BPMN 1.x has been so successful is that it "keeps simple things simple" by focusing on abstract business-level modeling, allowing developers flexibility in how to implement the technical details, and argues that BPMN 2.0 "should maintain this flexibility." In response, IDS Scheer's Sebastian Stein points out that a problem with BPMN 1.x is that it "only has implicitly defined execution semantics," and BPMN 2.0 needs to make them explicit. He goes on to neatly summarize the competing proposals:

It's my first time at SAPPHIRE, and I have one initial impression: this conference is huge. For me, 1,500 people at a conference is big, and this one is ten times that size. The press room is the size of a regular conference's general session ballroom. I just hiked 15 minutes to get to a session. More sessions run simultaneously than you'll find in total at most conferences. There are 30 official conference hotels. Wow. And I have to report that there are five bars of free wifi coverage everywhere in the conference center.

After a review of the massive schedule, I finally made it to a session: Ginger Gatling, SAP NetWeaver BPM Product Manager, giving an overview of the business process management (BPM) component in SAP, including a demo and some thoughts on the future functionality. She started with a discussion of the evolution of BPM, including the drivers that have moved us from the old-style workflow and EAI to the present-day collaborative design environment where multiple people might be working on modeling different components, from human-facing processes to rules. For SAP, however, a lot of this is future-state, not what they have now in the shipping product.

Regarding TIBCO's first-ever "analyst summit" at their annual TUCON user conference, I'll leave it to Sandy Kemsley to record the actual content of the presentations to analysts. I'll stick to the impressionistic view.

Apparently "the analysts" told TIBCO they wanted to hear executives talk about go-to-market strategy, so we got almost nothing about product and an awful lot about "value propositions." Are there really analysts who want to spend half a day hearing about value props and selling tactics? Scary. But, having lowered my expectations completely, TIBCO's "solution showcase" exhibits — open to the hoi polloi after the analyst event ended — actually blew my socks off:

Speaking at this week's "TUCON 08" TIBCO user conference in San Francisco, Christopher Ahlberg, founder of Spotfire and now president of that TIBCO division, discussed the capabilities of the technology and what's been done to integrate Spotfire into other TIBCO products.

Timely insight — the right information at the right time — is a competitive differentiator for most businesses, and classic business intelligence (BI) just doesn't cut it in many cases. Consumer applications like Google Finance are raising the bar for dynamic visualization techniques, although most of them are fairly inflexible when it comes to viewing or comparing specific data. In other words, we want the data selection and aggregation capabilities of our enterprise systems, and the visualization capabilities of consumer Web applications. Ahlberg sees a number of disruptive BI technologies transforming the platform — in-memory processes, interactive visualization, participatory architecture, mashups — and starting to be able to link to the event-driven world of classic TIBCO.

Perhaps you noticed that I renamed this blog "Competing on Decisions" from "Addicted to BI." Now I'll admit that the latter had a certain anti-chic appeal with its allusion to substance abuse, but frankly, I'm recovering from my BI addiction, so it's time to move on. I've also come (slowly it seems) to the conclusion that informing people (BI) is part of an incomplete cycle. If we as a company make an investment in BI, it isn't the informing of people that matters, it's what happens next. The decisions.

Last year, James Taylor and I wrote a book, "Smart (Enough) Systems," to make the case for creating "decision services" in order to automate certain kinds of high-volume, low-latency decisions. Dubbed EDM for enterprise decision management, we laid out the sort of reference architecture for getting this done, which included predictive modeling, business rules engines, some form of either process automation, or at least a smooth handoff to operational systems and back end analytics to both evaluate the quality to the decisions and manage some form of adaptive control of the decision models (in other words, test new models and compare to the results of the existing ones).

Oracle announced on Monday that it is entering the archiving market with the release of "Universal Online Archive." UOA positions Oracle to compete more directly against EMC and IBM in the e-mail and messaging archive space. It's interesting as only a year ago nobody was much interested in archiving, but in the past twelve months we have seen everyone from Dell to Google try to gain a foothold, and the market shows no signs of slowing down. It remains a chaotic and confusing sector risking a consolidation (which of course might not happen, or at least not soon).

UOA is built on top of Oracle 11g with technology acquired from Stellent, as well as from e-mail capture experts ZL Technologies. Why would Oracle be interested in archiving e-mails you may ask? Well the answer is simple: because there is an awful lot of it. And by archiving it, the messages will move out of Microsoft's servers and into Oracle databases. Remember in most firms e-mail is by far the single largest type of "data."

It seems my last post, drawn from a press release, keynote slides, and mini-briefing, missed the coded messages in IBM's business process management suite announcement. Here is the decoded version.

The announcement of an "IBM BPM Suite" represents a big deal internally at IBM. It is intended to signify a commitment to a single BPMS based on interworking components from separate divisions — WebSphere, FileNet, Lotus, Rational, GBS, etc. It required signoff from all the various warlords — Rosamilia, Goyal, LeBlanc, Bowden, etc. They know they're not there yet, but the commitment to get there is new.

There were lots of provocative questions and comments on my previous two posts ("Technology is Not the Driver of BI Adoption" and "BI and Technology: Part II"), so I thought I'd just batch all my responses together.

I attended IT360 this week, mostly to hear Matthew Glotzbach, director of product management for Google Enterprise. It's a sad commentary on the culture of Canadian IT conferences that this session is entitled "Meet Matthew Glotzbach of Google" in the conference guide, as if he doesn't need to actually talk about anything, just show up here in the frozen north — we Canadians need to work on that "we're not worthy" attitude!

Google's Enterprise division includes, as you might expect, search applications such as site search and dedicated search appliances, but also includes Google Apps, which many of us now use for hosting email, calendaring and document collaboration functions.

You're probably saying, "wait a minute, didn't IBM already have a business process management suite? Yes, I admit, they were in my 2006 BPMS Report series, in which they agreed (reluctantly, I hear) to let the combination of WebSphere Modeler, Monitor, WID, and Process Server be described as a BPM Suite. But here at IBM's Impact 2008 conference in Las Vegas, the company actually announced it has an orderable suite — sort of…

Last week, Lombardi held its second analyst update by teleconference; I found the first one back in January to be informative, and obviously Lombardi had sufficient positive feedback to continue. Strangely enough, we were instructed to embargo information about the new Blueprint until today, although the Blueprint team blogged about it on the weekend.

Phil Gilbert started out with a high-level corporate update, including growth — both new hires and through the channel — and some of the new sales where they continue to compete successfully against larger vendors. However, most of the information was about products and services.

We have space left in our two-day class Process Modeling with BPMN in Chicago on April 16-17. This is a great opportunity to jump-start your education on what has emerged as the important standard in BPM. The training is hosted by the BPM Institute and taught by me. This is the new v3.0 of the training material, based on BPMN 1.1, and includes 60 days use of what I think is the best BPMN tool around, Process Modeler for Visio from ITP Commerce. We use the tool for simple exercises in class, as well as for the certification exercises mailed in after class, with individualized feedback from me. That part is optional, but that's where you really learn how to do BPMN.

I'm having some problems with a March 20, 2008 article titled "Gartner: Emerging Technologies Will Help Drive Mainstream BI Adoption." This has been the Holy Grail of BI vendors for over a decade — to increase the number of "seats" using their products, widely reported to be about 20 percent of an organization but clearly much less than that. What troubles me the most about this article, or rather, about Gartner's analysis, is the supposition that new technology is going to crack this old chestnut. It won't. There are only two pieces of enterprise analytical software (broadly speaking) that ever gained currency in organizations in the past two decades — Excel and Google. Wouldn't it be a good idea to understand why?

Is there a business process management Hall of Fame? I don't think so, but there should be, to recognize the true pioneers and innovators in the field. BPM's core ideas and technologies come from several divergent fields, and my list would include those who first introduced them — ideas about what a business process is, and what managing one really means. Thinking about who should be in a BPM Hall of Fame is a fun exercise, and you might it helpful in framing your own views. My list emphasizes technology, recognizing those who first recognized that improving business processes demanded fundamentally new technology, often enabled by fundamental shifts in the surrounding IT environment.

My nominees would be:

BEA recently completed a "thorough analysis" of the business process management market, based on analyst reports, articles, and customer surveys. Some highlights, with my thoughts:

• BPM is one of the fastest-growing software markets, projected to go from $500 Million in 2006 to $6 Billion in 2011. When I see $6 Billion I have to wonder what they're counting, but yeah, it's definitely moving.

• Rapidly consolidating, from 150 vendors in 2006 to 25 in 2007. That's just silly. It was never 150, and it's more than 25 today. I would say the BPMS market is still ripe for consolidation, which hasn't really happened yet.

• 65% of BPM solutions in BEA's own survey integrate 3+ systems. A good sign I agree. Being BEA customers, though, I suspect that is well above the industry as a whole.

• Company politics and shortage of soft skills outweigh technical challenges. I agree, for a SOA shop, BPM is a piece of cake technically.

"Follow the money." This approach to investigation, applied by criminal prosecutors going back before Eliot Ness and made famous as a line in the movie "All the President's Men," is exactly how soon-to-be-ex New York Governor Eliot Spitzer was tied to a high-end prostitution ring. In this case it was fraud detection technology, of the kind routinely applied by banks in money laundering investigations, that led directly to Spitzer and to his resignation.

"Internal Revenue Service investigators conducting a routine examination of suspicious financial transactions reported to them by banks found several unusual movements of cash involving the governor," reported the New York Times in this story. "The transactions, officials said, suggested possible financial crimes — maybe bribery, political corruption, or something inappropriate involving campaign finance. Prostitution, they said, was the furthest thing from the minds of the investigators."

Philip Bierhoff, Systems Manager at Procter & Gamble, spoke at last week's FASTforward conference about strategies to increase user adoption as business intelligence goes mainstream [Editor's note: it's a topic very much at the center of Cindi Howson's recent feature on "Pervasive BI"]. P&G's Symphony project creates "decision cockpits": dashboards based on specific roles and corporate divisions, and including information ranging from traditional BI reports to documents to news.

The underlying data landscape has moved from their first iteration of a common data warehouse in the mid-'90s with regional servers plus ETL, storage and aggregation, where BI was driven by stored aggregations; to the current atomic data warehouse with a central server plus ETL and storage, where BI is driven by query rewrite — effectively, aggregation on the fly. They also have SAP generating data into SAP/BW; altogether, they have about 65 TB in the data warehouse and 50 TB in SAP/BW.

The subprime lending crisis offer fresh evidence that we're in the bear-skins-and-stone-knives era of understanding risk and making good decisions based on data. That's one of the key points I heard yesterday at an IBM Data Governance Council meeting in New York. As sophisticated as predictive models and enforcing business rules may seem, the technology is limited by a lack of best practices and standards and by the sheer scale and complexity of enterprises and financial markets. A first step toward avoiding such calamities, say Council members, is an integrated, overarching data governance program that addresses data security, data privacy and data quality so that risks can be better understood and outcomes anticipated.

"When the subprime loan scandal broke, a lot of people said, 'how could they not have known that they were sitting on billions of dollars of bad debt?'" says IBM's Steve Adler, who founded the 50-member Data Governance Council back in 2004. "The problem is that nobody really knows how to look at assets and liabilities and how decisions affect individual performance, the performance of divisions and the performance of companies. That level of institutional awareness about risk-based decision making does not exist."

"Discovery" is a legal process whereby parties to a lawsuit request and provide documents and information that may be pertinent in litigation. "Discovery" also describes an analytics goal that has nothing to do with the court system: extraction of useful information — data, facts, and rules, which together constitute knowledge — from databases and textual sources.

I had expected the December 2006 federal rules amendments on discovery of electronically stored information — "discovery" here in the legal sense — to open new vistas for application of knowledge-discovery technologies: data mining, machine learning, visualization, and the like. The reasoning is simple. Corporations must now retain vast volumes of electronic records including e-mail and information from enterprise operational systems. To comply with e-discovery mandates, they must be able to "produce" records in response to discovery processes, and that means metadata-management, classification, search, and similar systems.

At last week's Gartner BPM Summit, Elise Olding moderated a panel on weaving BPM into the enterprise, with Eric Abecassis, Architecture and Integration Manager with Schlumberger, Jim Boots, Enterprise Architect at Chevron, and Kevin Morgan, Program Manager at Dolby.

Abecassis started with the process-related problems that they had at Schlumberger: processes had to be standardized in order to effectively manage growth and improve execution, reduce the administrative burden on the field people, and improve alignment between business and IT. Their approach was to focus on three main types of activities:

Harvard Business Review recently hosted a two-day conference in Miami called Think!Analytics (not to be confused with the firm with the same name sans the exclamation point) featuring Tom Davenport and Jeanne Harris, co-authors of the current best-seller, Competing on Analytics. Some of you may remember that I was pretty tough on Tom when the article of the same name came out in the Harvard Business Review in January, 2006, but we've since mended our fences; so much so that I went to Miami to join the festivities and Tom has agreed to keynote the Enterprise Decision Summit in October which James Taylor and I are co-chairing.

Speaking at this week's Gartner BPM Summit in Las Vegas, Jay Simons, VP of Marketing for BEA, presented the company's recent research results on the state of the BPM market, including a survey of 200-plus BEA customers, mostly IT people but spread across vertical markets and geographies. They've also gathered information through their online BPM Lifecycle Assessment.

The results show a number of interesting trends indicating that CIOs and business leaders are focused on improving their processes. Existing customers described how they expect to get their ROI from their BPM implementations, and most expect to see ROI over the next three years.

I'm here in Vegas for Gartner's 5th BPM Summit, and they're reporting about 1,000 attendees (though I'm not sure if that includes Gartner and vendors). For those of us who attend business process management events religiously, I'm hoping it's not a complete replay of September's BPM Summit in Orlando.

Janelle Hill gave us Gartner's big-picture view of BPM, which will be covered in detail in other sessions throughout the conference. Hill seems to be hitting her stride as Gartner's face of BPM since Jim Sinur left almost a year ago. She started with the now-familiar view of process improvement over the ages, from Deming and Taylorism through TQM, BPR, Six Sigma and a variety of other methodologies and tools since the 1920's. This has changed from a focus on scientific management, to computerized process flow, to package applications as best practice, to flexible and adaptive process.

The signs are familiar and worrying: a US economy that cannot seem to rebound, job losses on the rise, and consumers getting increasingly jittery. Will US companies, in a desperate bid to cut costs, intensify their push to send work offshore? Not so fast.

As I land in India's economic capital Mumbai (erstwhile Bombay) on a work and personal trip, headlines in The Economic Times — India's answer to The Wall Street Journal — present a contradictory picture about the opportunities for the Indian offshore industry. Here are some representative news items.

Lombardi held an analyst conference call last week in advance of today's press releases — a relatively new format for them — to discuss their executive reorganization against the backdrop of their 2007 results and 2008 strategy. Rod Favaron, CEO (and, until last week, President) and Phil Gilbert, President (formerly CTO) gave us the update. The press releases are here and here.

Despite this country's credit crisis, it appears there are still plenty of big-money bets being placed on emerging information technologies. This morning alone I've seen hefty sums put into data warehouse appliances, complex event processing (CEP) and master data management (MDM). Knowing a thing or two about each market, I'd say they are far from subprime investments.

Are you an Oracle or BEA customer trying to make sense of the combination? Forrester's John Rymer and Mike Gilpin have written an extensive analysis of the overlaps and of which products are likely to prevail. Oracle is promising long-term support for BEA products whether they're continued or not. But eventually, say the authors, "carrots and sticks" are likely to prod customers toward the preferred, go-forward products.

The big news today is that Oracle is buying BEA. Everyone saw this coming, but I offered my take on "Why Oracle Needs This Deal" last October. The question is, what will Oracle do with BEA, and how will this help or hurt BEA and Oracle customers?

So EMC (read: Documentum) acquired Document Sciences. The announcement came over the holiday period and has been the topic of chatter in the blogosphere. It's an acquisition that makes perfect sense for EMC as it continues to reposition Documentum away from the traditional complex document management activities that established the firm — a market that is under attack from Microsoft and Open Text — and more into high-value, transactional document management and archiving.

As 2007 came to a close, Gartner issued its long-awaited 2007 Magic Quadrant on Business Process Management Suites (BPMS). Gartner's previous BPMS Quadrant was issued in June 2006, so nearly 18 months had lapsed in its review cycle. No matter, 22 BPMS vendors — or at least the ten in the top-right quadrant — now had reason to celebrate. Or did they? It wasn't long after the Dec. 14 publication of the report (available from Quadrant leaders Pegasystems and Lombardi) that I received a call from an irate vendor.

I was recently asked to review a business process diagram that was intended to capture current state for a service disruption planning process. I quickly found out that the challenge was not so much assessing the diagram itself, but resetting expectations of users that seemed to be already sold on the diagram, despite its numerous deficiencies. This is a classic pitfall in business process analysis (BPA).

As the year winds down I'm in a reflective frame of mind. Today I posted the list of IE's Top 20 Articles of 2007. It's an interesting indication of reader interest, but being measured in page views, the list doesn't do justice to all the single-page blogs we publish. (On the other hand, if a reader clicks to the very last page of a multi-page article, they're truly engaged!) Thus, for those who follow our blogosphere, here are the Top-20 Intelligent Enterprise Blogs of 2007:

"As you may know, Intelligent Enterprise ceased print publication with the January 2007 issue, but rest assured that the mission lives on and is being reinvigorated here at IntelligentEnterprise.com… "

I posted these words nearly a year ago, and I'm happy to report that we've delivered on what we promised — up to a point.

In the early days of BPM — four or five years ago — everyone thought BPEL was the BPM standard, at least for runtime execution. Not long after, the importance of business-friendly process modeling came to the fore, and BPMN emerged as the standard for that. The mismatch between graph-oriented BPMN models, where you can route the flow just about anywhere, and block-oriented BPEL, where you can't, didn't seem to worry BPM vendors. After all, a model was just a model, a business requirements document in diagrammatic form. The BPEL designer would use the BPMN as business input to the implementation and go from there.

Then a new concept emerged, the BPM Suite, which included process modeling, executable implementation, and BAM in an integrated toolset that promised the improved business-IT alignment and agility needed to cope with ever-changing business requirements. Suddenly the process model became more than a business requirements spec. It was actually the first phase of the process implementation. No problem, said the BPEL vendors. We'll just generate skeleton BPEL from the process model, and use that as the starting point for the BPEL designer. Voila! Business empowerment! Business-IT alignment!

In the early days of BPM — four or five years ago — everyone thought BPEL was the BPM standard, at least for runtime execution. Not long after, the importance of business-friendly process modeling came to the fore, and BPMN emerged as the standard for that. The mismatch between graph-oriented BPMN models, where you can route the flow just about anywhere, and block-oriented BPEL, where you can't, didn't seem to worry BPM vendors. After all, a model was just a model, a business requirements document in diagrammatic form. The BPEL designer would use the BPMN as business input to the implementation and go from there.

Then a new concept emerged, the BPM Suite, which included process modeling, executable implementation, and BAM in an integrated toolset that promised the improved business-IT alignment and agility needed to cope with ever-changing business requirements. Suddenly the process model became more than a business requirements spec. It was actually the first phase of the process implementation. No problem, said the BPEL vendors. We'll just generate skeleton BPEL from the process model, and use that as the starting point for the BPEL designer. Voila! Business empowerment! Business-IT alignment!

I sat in on a presentation by Michael zur Muehlen on business processes and rules at the recent IIR/Shared Insights BPM conference. Michael is responsible for Business Process Management courses at Stevens Institute of Technology. He started out with the bottom line on why you want to integrate process and rules:

• Simpler processes
• Higher agility
• Better risk management

Who wouldn't want this? However, he points out that users don't like processes, since they find them abstract (or possibly requiring a more analytic view of the organization) and restrictive (that is, not able to capture all the actual business cases). He also points out the obvious problem with Eclipse-based process modeling tools: they're not friendly to business types, so you end up with technical people maintaining business processes, which usually results in a lot of code and the next generation of legacy systems.

In "Measuring and Managing Performance in Organizations," Dorset House Publishing, 1996, Robert Austin made a very clear case that performance measurement often leads, paradoxically, to distortion and dysfunction instead of improvement. According to Austin — and I agree with him, having witnessed this phenomenon firsthand more than once — measuring an indicator of a performance (since we usually can’t indicate the actual performance itself), raises the risk of making things worse. How can that be?

When I launched my course "Process Modeling with BPMN," I discussed why so many people beginning to "do" business process management (BPM) were looking for training in modeling, and why that was especially needed for BPMN. Now, having delivered the training, I have a better appreciation of Business Process Modeling Notation's strengths and limitations, a better understanding of what students really want, and what they really need to know about BPMN modeling. This post describes what I got right the first time and where I've had to adjust.

The consolidation game continues apace, this time in the Business Rules solutions marketplace. I received an e-mail message from Paul Haley, the erstwhile founder and chairman of Haley Systems, one of the better-known business rules software in the US, informing me that Haley Systems had been acquired…and that, not having gone with the acquisition, he himself is now once again foot loose and fancy free ("like you, vendor neutral," as he put it).

For the record, Haley Systems was acquired by RuleBurst, which is an Australian firm with a good footprint in areas of the world that Haley Systems did not cover, so there is now another truly global player in the business rules market.

Mark Smith's Friday blog post was just the latest in a chorus of calls for IBM to drop the above-it-all attitude and jump into the business intelligence market. Well, the company has finally responded, announcing this morning that it will acquire Cognos for $5 billion in cash.

What gets me about IBM is that it is such a cool cucumber. Here it is, the last to act with few good choices left, yet it manages to come up with a winner, painting it as a carefully considered deal it came up with after scouring a vast array of choices.

I'm sure that James Taylor has almost given up on me ever writing a book review of Smart Enough Systems: I wrote a brief advance review back in April that's printed in the book, but nothing since it was released. Recently, I had a chance to finally meet James face-to-face after a couple of years of emailing back and forth. Also, James' situation has changed since the book was released: he left Fair Isaac and is now an independent, working (I think) with his co-author, Neil Raden. Neil, who also I met briefly recently, is an independent consultant and analyst who has been focused on business intelligence for quite a while; James refers to his work as "BI 2.0″ (a term that I think that I invented in my blog in early 2006). The two of them met through James' blog and started the conversation about how someone needed to write a book about this crossover area between business rules and business intelligence.

Just to get started, here's my pre-release review:

At last week's Business Rules Forum, Kathy Long of the Process Renewal Group talked about how to derive rules from processes and use them as guides to the process. There are a number of process-related problems that can occur when the rules are not explicit: assumed policies, activities with experience as the only guide, and inconsistent (and therefore likely non-compliant) processes.

The key things to consider when analyzing the guides for a process can be focused around what happens at a given activity (and what knowledge is required, what decisions are required, what reports have to be generated) as well as a number of other factors. Long presents a number of questions to ask to drive out the rules and make them explicit.

After rejecting Oracle's initial bid for $6.7 billion, BEA has now indicated readiness to sell itself to Oracle ("or any other bidder") for about $8.2 billion. With Oracle's insatiable appetite, BEA's relative stagnation, and incessant pressure from billionaire investor Karl Icahn on BEA management, the acquisition now seems to be only a matter of time. So, what does Oracle get for $8 billion and change? A cake that it has always coveted…and an icing to die for.

Oracle has always been a leader in terms sheer market size and growth, but the same can hardly be said in terms of its technological prowess. The fact is, Oracle has always lagged behind its acquisitions — be it Siebel (CRM), PeopleSoft (HRM, Hyperion (BI) — or now BEA (application middleware). On the other hand, BEA has always led the market in terms of technology. Consider, for example, its main product lines around the application server (Webogic), integration services (AquaLogic) and transaction processing (Tuxedo).

David Straus of Corticon gave an engaging presentation here at this week's Business Rules Forum about business rules and business intelligence, starting with the Wikipedia definitions of each. He characterized BI as "understanding" and BR as "action" (not unlike my statement that BI in BPM is about visibility and BR in BPM is about agility). He started with the basic drivers for a business rules management system — agility (speed and cost), business control while maintaining IT compliance, transparency, and business improvement (reduce costs, reduce risk, increase revenue) — and went on to some generalized use cases for rules-driven analysis:

• Analyze transaction compliance, i.e., are the human decisions in a business process compliant with the policies and regulations?

• Analyze the effect of automation with business rules, i.e., when a previously manual step is automated through the application of rules

• Analyze business policy rules change (automated or non-automated)

The ongoing sub-prime mortgage crisis has already taken its toll in more ways than one, and it's now adding a new twist to the concept of data ownership. For all of us sitting back thinking it's someone else's problem, here's a real eye-opener.

American Home Mortgage Investment Corporation (AHM), among America's biggest mortgage lenders, recently filed for bankruptcy on account of its sub-prime lending exposure. This, in turn, alarmed large AHM clients like Freddie Mac and Ginnie Mae, which promptly terminated client-servicing agreements with AHM and asked it to return client files, meaning mortgage files for individual home owners serviced by AHM on their behalf, including data related to mortgage principal, interest, property taxes and insurance (PITI).

After a brief intro by Gladys Lam, the executive director of the Business Rules Forum, the conference kicked off with a keynote from Ron Ross, the driving force behind this event and a big name in the business rules community. A couple of things are distracting my attention from his talk: I'm up directly after him, and I'm presenting in this room, which is the main (read: big) conference hall. Let me make my ever-present complaint about passworded wifi in the meeting room and no free wifi or wired internet in the hotel, since I know that my regular readers would be disappointed without that news from the front lines.

Ron and I have exchanged email over the years, but this is our first opportunity to meet face-to-face; I'll also have the chance to meet James Taylor and a few others