Intelligent Enterprise

Better Insight for Business Decisions

RSS
Webcasts
Whitepapers
Subscribe
Home

March 8, 2002

The Confidence Game

Giving your users access to "enhanced" metadata in strategic business apps can increase their confidence in making important decisions

By Karen Kazimer-Shockley

Continued from Page 1

However, the business rule decision-making process was not yet over. What should happen if the birth date was not present, contained invalid characters, or listed the customer as more than 100 years old? Should the record by discarded? Should a code be entered into the birth date field? Should this code be different if the date was invalid as opposed to not present?

In this instance, the users determined that their demographic database had several old records that should have been deleted. They took this opportunity to delete (or actually, not load) those records for customers who were over 110 years old. They also determined that they would enter 01/01/1000 for records that had invalid or no data.

These business rules also captured any processing that varied according to the calendar year. (See Table 1.) At times, business processes changed; for example, if a delivery occurred before 1995, it was recorded in a different way.

We captured these business rules in the same database used for the data warehouse; the users had access to both the data and the metadata through a single interface.

PROCESSING STATISTICS

Many business decisions are made using data that is incomplete or varies in currency, so the second step in creating high-value metadata is to give users statistics about when and how often the data is delivered. For example, you could indicate the number of records received and processed (as well as the dates involved) for each site.

In our project, we captured this data through added modules in the architecture's extract-transform-load layer, so it was relatively cost-free to the client. However, the rewards were huge. First and foremost, users could determine if they were making decisions with the latest data available. They could also determine if they had complete data because they knew by site and date which data they had. Furthermore, they had insight into the data's quality, based on the number of records per site, by date, that were rejected.

This high-value metadata can also be used to spot trouble areas and deploy new operating procedures and software or hardware to improve the receipt of data from the source systems. Additionally, it can be used to compile trends, so that users can know if certain sites are moving toward providing more current, quality data.

For example, in the sample processing statistics shown in Table 2, you can see the following:

The national systems appear to have higher percentages of processed records. This can be due to several circumstances; for example, because these systems send feeds on a monthly basis, they're less likely to be volatile. (Records have a chance to stabilize prior to the feed.) In addition, the sales and demographic systems may be more mature. Maturity might indicate that most of the data entry anomalies have been addressed, as well as that users may be thoroughly trained in these systems.
The delivery sites have lower percentages of processed records as a group. Because this data is loaded daily, it's more likely that later updates can correct existing deficiencies. Furthermore, it appears that the last site, in Raleigh, NC, has a much lower percentage of quality records than the other sites. This may be because Raleigh is a new site, or that there has been a large turnover of staff. If the trend continues, management might consider investigating this concern and generating a plan of action.

As an added benefit, along with this information, you could include how often the data is received from each source system. For these users, the demographics and sales system provided monthly feeds, while the delivery data came daily. Again, from Table 2, you can see that two of the sites, Biloxi and Raleigh, appear to be a day behind in their transmissions. If they are consistently behind, you might want to investigate their communications software and hardware or perhaps their processing schedules.

METADATA APPLICATION

The third part of the process is to create an online application that allows everyone involved with the data decisions to have access to the metadata.

The business rules are captured during the requirements phase and should be made available to modelers, designers, and developers, who, in turn, update them as new information is learned about data content. (Very often what the user thinks is the data domain needs to be expanded).

The metadata continues to be updated as it passes through the software development life cycle, including the maintenance phase. An added benefit to the users is that as they add functionality or interface to more systems, they have current definitions of the data elements that they are using.

Giving users insight into the applied business rules provides greatest value during the decision-making process. The most obvious benefit is to let them know what 01/01/1000 means when they see that information in the birth date field. They know that it's specifically inserted because it has been determined that some value can be obtained from that record, even though the birth date is incorrect.

A second benefit is to increase their trust of the data integrity. If they query the customer acquisition system and the new data warehouse, they'll get different answers. However, they'll also know that all the erroneous records have been deleted.

A third benefit is that they can understand the timeliness of their data. It you're a user, it can be very helpful to know whether you're making decisions based on data that is four months old or that was delivered yesterday.

Rate This Article

Comments:

Optional e-mail address:

The metadata application I've described here can be integrated into the user interface tool. In the case study I described earlier, we created an interface into a Business Objects online analytic processing system that users could access by a right click of the mouse. In another instance, the high-value metadata application was available via a Web interface.

By implementing high-value metadata at the beginning of the requirements process and carrying it through to delivery in the user interface, you can significantly help users understand the meaning, the currency, and the completeness of their data. And because users are involved in the creation of this metadata from the beginning, they have a significant stake in ensuring that it's updated as new business rules and new data sources come into being.

Karen Kazimer-Shockley [k.kazimer-shockley@att.net] is a senior consultant working for Systems Management Engineering Inc., of Reston, VA. She has more than 25 years' experience in IT, spanning mainframes, minis, client/server applications, and Web-based products.

RESOURCES

Tannenbaum, A. Metadata Solutions: Using Metamodels, Repositories, XML, and Enterprise Portals to Generate Information on Demand. Addison Wesley, 2001

Related Article at IntelligentEnterprise.com:

"Ghost in the Machine," Dec. 5, 2001