Pillar of the Community
XML is becoming the standard platform for the interenterprise processes on which B2B e-commerce depends
By William J. Lewis
The trade press calls them “the next big thing.” Forrester Research says they’re “all the rage.” They’re business- to-business (B2B) exchanges, also known as e-marketplaces or online exchanges, a growing class of Web portals that facilitate buying and selling interactions among a community of organizations. In essence, they’re the trading floors of the New Economy.
In their recent book B2B Exchanges: The Killer Application in the Business-to-Business Internet Revolution (ISI Publications, 1999), Arthur B. Sculley and W. William A. Woods distinguish these sites from other B2B e-commerce enablers by pointing out that the former involve multiple buyers and sellers, centralize and match buy and sell orders, and provide post-trade information. In a January 2000 report called Business Models for the New Economy, my firm — Cambridge Technology Partners — describes B2B exchanges as “Internet-based companies [that] have appeared expressly to facilitate the exchange of value between discrete value-creating entities that exist both upstream and downstream in the value chains of complex industries.”
This “value” is largely vested in the form of data. Thus, B2B data integration is a strict requirement of this process: matching buy and sell orders requires many-to-many mapping and transformation of digital transactions among various formats. In the past, these processes involved tedious, manual development, but now numerous software vendors, taking several differing technical approaches, offer packaged solutions to support these capabilities. Not surprisingly, the common thread among these approaches is the use of extensible markup language (XML) as a universal data source, destination, storage format, and in some cases, transformation language. We can classify these approaches into four overlapping categories: enterprise application integration (EAI) extensions, transformation hubs, XML-database interfaces, and XML transformation-management utilities.
From an architectural viewpoint, B2B data integration is closely related to EAI, also known as application-to-application integration (or A2A, the term I’ll use here). However, they have at least two important differences. First, in B2B data integration, the applications involved run inside the firewalls of two or more companies; in A2A, they run inside just one. Second, B2B technologies increasingly leverage XML and extensible style language (XSL) for data integration, in addition to the process-integration support via message brokering and workflow automation typical of A2A solutions. (See my previous article, “Forging the Value Chain,” Intelligent Enterprise, January 20, 2000.) Until recently, most data integration projects have focused on intraenterprise goals; that is, doing A2A within a single enterprise. But with the rapid proliferation of external-facing applications such as B2B exchanges, the focus of data integration is now rapidly moving toward interenterprise applications. In such applications, transforming and moving data precisely and efficiently is critical to not just a single enterprise, but to groups of enterprises— all the trading partners involved. And XML will be the standard input and output language of these data transformations.
Staking New Ground
The established A2A vendors now recognize this opportunity and are joining with relative newcomers to swarm the B2B data integration space like ants on a picnic. The most visible players include Acta Technology Inc., Mercator Software Inc., Vitria Technology Inc., Extricity Inc., and Computer Associates International.
Not surprisingly, most of the A2A software providers are taking a B2B approach that leverages their existing product architectures. The B2B Mapper component of Active Software Inc.’s Business Exchange Server, for example, supports the generation and consumption of XML documents via transformations between XML documents and ActiveWorks events. (In ActiveWorks, an “event” is a message carrying a predefined set of data elements.) For example, an XML order document — carrying customer, product, and amount elements — could be transformed to an ActiveWorks event for subsequent routing to an inventory application. This architectural approach is similar to the XML interfaces supported by RDBMS and “XDBMS” products. A more “lightweight” alternative, however, is to map and transform data among just XML documents — a strategy implemented by B2B transformation hubs. These hubs provide a neutral, many-to-many focal point for the external interfaces of buyer supply chains and provider delivery chains. Standalone, full-function transformation hubs are on the cutting edge, and are either in beta or have just recently been made available. Examples include Microsoft’s BizTalk Server, in beta as of this writing; IBM’s WebSphere Commerce Suite Marketplace Edition, scheduled for release in Q3 2000; Intershop Communications’ Marketplace Toolkit; Tron, under development by Allaire Corp.; and OnDisplay Inc.’s (recently acquired by Vignette Corp.) CenterStage eBizXchange.
Because B2B exchange partners will probably use disparate formats even within the same transaction type, a vehicle for rationalizing and transforming these various formats is critically needed. Given the numerous existing and possible standard and nonstandard data interchange formats — EDI, X12, SWIFT, delimited, positional, multiple XML document type definitions (DTDs) and schemas, and so on —establishing an “any-to-any” hub transformation architecture would yield an unmanageable Cartesian product of mappings.
Enter XSL
One practical approach for managing many-to-many transformation multiplicity is to reduce the problem down to a common denominator: transformations between any two XML schemas.
You may remember the “universal translator” of Star Trek fame: XSL may become the universal translator within B2B exchanges. XSL offers a standard, declarative, and largely self-documenting format for specifying transformations of XML data. In essence, it serves as an XML vocabulary specialized for transforming or formatting XML documents. Another XML vocabulary, XSL Transformation (XSLT), is designed specifically for transforming XML documents into other formats. Many people accustomed to venerable transformation tools such as Informatica Corp.’s PowerMart or Microsoft SQL Server Data Transformation Services (DTS) may not yet intuitively understand the syntax of an XSL style sheet (which, by definition, is also an XML document). But just as XML is taking its place among the major languages of e-commerce, XSL will play a pivotal role in translating the various dialects in which e-business events are articulated and stored. For that reason, I highly recommend that you become thoroughly familiar with XSL as well as XML. (See Resources.)
XSL Transformation In Action
BizTalk Server 2000, which Microsoft calls “a data translation and application-integration server” for exchanging business data, is a good example of an XML-based B2B data transformation hub approach. Its BizTalk Mapper component uses XML and XSL to map and translate data for B2B application integration.
Say a buyer wants to send a purchase order to a seller through a B2B exchange, and each party uses a different standard format for purchase orders. Figure 1 shows a high-level overview of how BizTalk Server maps and transforms the purchase order from the sender’s format to the receiver’s format. This type of architecture should look somewhat familiar to anyone experienced in data warehousing or EAI efforts.
FIGURE 1 B2B exchange data transformation architecture.
The top of the diagram shows components that you can develop using BizTalk Server Editor and BizTalk Server Mapper. (See Figure 2.) The source and destination specifications are specialized XML schemas, developed using the Editor, which describe the structure of the source and destination documents. The XSL transformation map is an XSL style sheet that describes the mappings and transformations necessary to create the destination from the source. The transformation map indicates the required transformation functions.
FIGURE 2 BizTalk Mapper.
The lower section of the diagram indicates the runtime flow. In this example, the buyer sends an XML document to the exchange, referencing a previously defined BizTalk Server information- exchange agreement with a designated selling partner. The transformation map indicated in the agreement is applied, then the map uses the source and destination specifications and the appropriate functions to translate the source document into the target document. The target document is then forwarded to the seller.
In the top section of the screenshot in Figure 2, the source specification appears in the lefthand side of the window, and the destination specification in the righthand side of the window. The workspace between them shows mappings among fields in the source and destination, as well as transformations — “functoids,” in Microsoft parlance — that occur between source and destination. The lower section of the screen shows the XSL style sheet code generated by the mappings and transformations, as well as the “functoid palette” from which you can choose various transformation templates, including string, math, logical, date, and custom VB script.
As a point of comparison, Figure 3 shows a screen from SQL Server DTS. Both products map between source and destination layouts and specify transformations. The contexts may differ — data warehousing vs. a B2B exchange — but to a great extent, data transformation is a pervasive process. One of the most significant differences between these two tools, however, is that BizTalk Mapper generates more or less self-documenting transformations (XSL stylesheets), arguably better documentation than what you can currently extract from SQL Server DTS.
FIGURE 3 SQL Server Data Transformation Services; note the similarity to BizTalk Mapper.
Microsoft’s simplified object access protocol (SOAP) is functionally equivalent to BizTalk in that it lets distributed applications communicate over HTTP using XML-based messages. (In fact, BizTalk extends SOAP in several ways.) Consider a hypothetical purchase order XML document, submitted in a reverse-auction scenario to a number of potential suppliers. When someone inputs the purchase order, the buyer’s order-entry system generates a SOAP message containing a “payload” that encapsulates the PO document along with the appropriate process name (SendOrderForBid, for example). The SOAP message then transmits via HTTP to a B2B exchange, and the exchange server (appropriately configured) would receive the message, execute the specified process, and return an appropriate response to the buyer’s application.
Database-Centric Solutions
Most of the end-points for B2B exchange interactions — sources and destinations — will be application databases. These established databases are almost exclusively relational; however, XML is developing into a viable, native storage format. Venerable relational database providers as well as emerging XML vendors are offering facilities you can use to transform stored data into and out of the “standard” B2B exchange format (XML). RDBMS products following this trend include Microsoft SQL Server 2000 and IBM’s DB2 v.7; newer XML-centric database management system (XDBMS) solutions include Excelon B2B Integration Server from Excelon Corp. and the Adabas-oriented Tamino from Software AG.
Because a typical XML document is a hierarchical, rather than normalized, structure, transforming between XML structures and relational databases has challenged the DBMS vendors. Approaches for solving this problem include providing XML-specialized input and output functionality, new datatypes, and entirely XML-native storage structures.
IBM’s XML Extender for DB2, for example, includes XML-specific stored procedures, a document access description (DAD) construct enabling document-to-database mapping, and an XMLVAR CHAR column datatype for storing entire XML documents in table columns. Microsoft SQL Server 2000 also includes functions that make the database both a producer and consumer of data in XML formats: Transact-SQL keywords are available that specify the rendering of result sets in XML format. For example, you could submit the query:
SELECT+firstname,+lastname,+home phone,+title +FROM+Employees+FOR+XML+AUTO
|
TWO HEADS ARE BETTER THAN ON
|
|
Two recent merger announcements in the e-business software industry indicate a steady trend toward consolidation. On the same day, May 22, 2000, B2B integration software vendor WebMethods Inc. announced its intention to acquire A2A vendor Active Software Inc., and B2C specialist Vignette Corp. announced a merger agreement with B2B vendor OnDisplay Inc. The WebMethods-Active Software consolidation will create a deep B2B integration solution that spans the corporate firewall — front, back, and in between. This union has the potential to create a complete package that will be attractive to many B2B enterprises, including exchanges. On the other hand, the Vignette-OnDisplay agreement — although resulting in a single solution for creating delivery and supply chains outside the firewall — will do little to help connect the chains within the enterprise. Companies seeking a complete integration solution will need more robust A2A capabilities (such as those offered by ActiveWorks).
|
|
as part of a URL. The “+” signs represent spaces in the URL; FOR XML indicates that the result set should be returned in XML format; and AUTO indicates that the format should specifically be a simple, nested tree structure. Adding the XML DATA keyword to the end of the statement would return an XML schema for the result document as well. As you can see from this example, XML-extended database queries will enable relatively straightforward request initiation and results delivery over the Web.
SQL Server 2000 also supports new XML-specific stored procedures. For example, sp_xml_preparedocument parses an XML document into an intermediate, in-memory tree structure. This intermediate parse tree can then be referenced using the OPENXML keyword, allowing SELECT access to various levels in the in-memory representation of the document, and renormalization (what Microsoft terms “shredding”) of the document via INSERTs to appropriate database tables.
In contrast, in the emerging XDBMS market, Excelon’s B2B Integration Server is based on a native XML database. It supports business-process workflow automation, and includes a B2B translator for XSL-based translation of documents among various XML dialects. Software AG’s Tamino not only stores data in XML format, but also enables connectivity to non-XML data sources such as RDBMSs. This feature lets users query heterogeneous data via extensible query language (XQL) and receive result sets in XML format.
B2B Exchange Analytics
As B2B exchanges grow more competitive, they will aggressively seek new ways to squeeze revenue over and above transaction fees from their copious and accruing data assets. One means for doing so — which I predict will become increasingly common among B2B exchanges — is the offering of post-trade information, or analytic business intelligence services, to their constituents. To accomplish this goal, the exchange providers will have to capture and aggregate all the business events for which a B2B exchange is a conduit — bids, quotes, orders, settlements, and so on — for analysis.
To date, the relational model remains the standard language with which persistent data is organized, as well as the language understood by most tools organizations use to access, manipulate, and analyze persistent data. Thus, in the near term, support for business intelligence services will probably require transformations of XML documents into relational structures, and vice versa. The capture and translation of XML documents flowing through a B2B exchange into relational structures may allow for more direct application of commodity analytic software to this data. In this approach, the provider could render result sets from analytic relational queries as XML documents for subsequent distribution to trading partners. However, native XML database products such as Excelon and Tamino may mature to the point of giving RDBMS vendors a run for their money in supporting analytic applications.
Transformation and Management Utilities
The transformation products I’ve discussed to this point are components of larger e-business suites supporting a range of e-business functionality. We can classify another class of lower-profile, more specialized products as XML transformation and management utilities. Data Junction Corp.’s XMLJunction, for example, leverages the data transformation underpinnings of the venerable Data Junction tool to automate the transformation of numerous types of input formats into XML documents. Other utility- type tools provide specialized support for managing and optimizing XML data and metadata.
Data Volumes and Performance
XML is a relatively verbose data representation: An XML document, in contrast to a record-based rendering, includes field names (tags) for every field on every instance in addition to data values. This fact has implications for storage capacity requirements as well as for transmission and processing performance.
For example, here’s a size comparison of a small data set rendered in various formats. The implications on larger data sets are easy to extrapolate: KB Relative Size Text, Delimited 130 1.0 Text, Positional 231 1.8 XML 273 2.1
As with any data-intensive application, you must seriously consider such issues when constructing or maintaining scalable B2B sites. As an example of XML transformation performance, Microsoft defines “optimal performance” for BizTalk Server 2000 transformation throughput as 10 to 14 documents (transactions) per second.
XMLZip, available from XMLSolutions Corp. as a free download, addresses the verbosity problem by allowing users to compress specific nodes in the tree structures of XML documents. Other utilities from XMLSolutions include an XML schema-management tool (Schema Central) and the XEDI Translator tool, which dynamically translates EDI transaction formats into their corresponding XML schemas.
Data Management Opportunities
The rise of B2B exchanges, as well as other B2B applications, has extended the need for competent and proficient data management beyond the firewall, and will require the increasing, committed involvement of data-management professionals. You should aggressively ramp up your assimilation of these tools and techniques and seize every opportunity to contribute to B2B exchange initiatives.
William J. Lewis (datamodel@aol.com) is an associate director in the analytic business solutions practice of Cambridge Technology Partners. He has more than 20 years of experience in IT.
|
|
Subscribe to the newsletter
|
|
