Intelligent Enterprise

Better Insight for Business Decisions

RSS
Webcasts
Whitepapers
Subscribe
Home

May 31, 2003

It's All Subjective

Which, if any, is the right textual business intelligence solution for your company?

by Jeanette Burriesci

Continued from Page 1

Recommind CEO Bob Tennant stresses that his company's entity extraction capability is completely language-independent. One of his star customers, the Research Libraries Group, applies Recommind's MindServer to its document base representing 365 languages. Yes, that many languages exist.

Tennant says this customer partly inspired the company's last scalability expansion. It can now handle tens of millions of records. The Research Libraries Group has 40 million.

This newer approach to document categorization replaces human effort. But by gaining back personnel expense and time, do you have to sacrifice accuracy? Is this another trade-off? Tennant claims that tests of his system against human experts have shown MindServer to be not only at least 3,600 times faster, but also more accurate.

Is This For Real?

You have to ask yourself whether you can believe such outlandish claims. The bad news is, no industry benchmarks exist to corroborate any textual BI vendor's claims. Some standard tests do exist, however. The National Institute of Standards and Technology holds a competition every year at its Text Retrieval Conference (TREC). TREC uses academic measures of recall and precision to judge competitors. But the TREC tests, says Gartner analyst Alexander Linden, are done on text bases that are specific to the government vertical.

If you want to know how textual BI products perform on data in your industry, you can try talking to existing customers in your space. But even if you can rely on that information, say both Linden and analyst Nick Patience at The451, it still may not indicate how the software will perform at your individual company.

Both analysts go on to say that each company has its own very specific needs, all of which are crucial to textual BI software selection. Internal processes, potential users, desired results, and estimated software costs, are all at least as important as the domain the documents refer to.

Rate This Article

Comments:

Optional e-mail address:

Therefore, industry benchmarks for textual BI software performance are unlikely. Sealing the fate of such benchmarks, Linden and Patience agree that companies consider their performance tests to be valuable intellectual property, and wouldn't share them with an industry group. Pilot projects that demonstrate performance specifically on your data and help you gauge user buy-in are apparently the only way to go.

It's probably the complexity and uncertainty of selecting this type of software that leaves many companies to buy their solutions from "No Decision Inc.," or "NDI" — what textual BI vendor Antarctica Systems Inc.'s CEO tells Patience is his toughest competition. But, as these vendors strive to market their technology to specific verticals and concentrate on ease of use, NDI will increasingly lose business.

Jeanette Burriesci [jburriesci@cmp.com] is senior editor of Intelligent Enterprise.