Weighing the Evidence

In today's business world, data is very often guilty until proven innocent

Telling the story that's buried in the data" is admittedly a somewhat simplistic way of referring to corporate-level data analysis. But it's a phrase that captures both the essence and value of the analytic process.

Improvements in technology have made large-scale data analysis more practical for skilled and well-equipped specialists. However, a broad range of skills beyond the technical are often necessary to do this job right. In particular, the ability to effectively express the results of an analysis task is critical to achieving the end goal - helping managers make better business decisions faster and with less risk.

The Key Ingredient

When you distill this process down to the most meaningful level, a coherent end deliverable - an easy-to-read and reliable reporting package - is crucial for a successful data analysis project. And this deliverable must indeed tell a story for the primary reader and others who must understand and leverage its content.

Data acquisition is often the adventure part of the story, as told in the deliverable, because data acquisition can be both expensive and intimidating. Data often resides in data warehouses or in a patchwork of legacy systems - and sometimes includes purchased third-party sources. As the supply of data rapidly increases, so do management expectations for leveraging this resource to improve the precision and speed of their decision-making processes.

If circumstances are pressing you to move ahead with a strategic business decision, but you can't get trustworthy data quickly enough, you must ask yourself, "How willing am I to make an uninformed or potentially poor business decision?"

The consequences of not having enough decision support - and not being able to get it fast enough - are costly and come with various possible risks. Corporate managers face this jeopardy on a regular basis. Here are a few examples in which the timeliness and accuracy of data play a crucial role:

• Strategic business decision support. All industries must make corporate-level decisions that require "getting the story told." In these instances, time definitely equates to money. One example is getting an accurate profile of customer purchase patterns to update a Web site storefront. Another is reviewing payment streams to identify opportunities for potential cost recoveries.
• Critical event situations. If your organization is experiencing a merger or acquisition, litigation, fraud investigation, or a regulatory compliance issue, data analysis accuracy and speed can be crucial. In an acquisition, for example, an electronically generated customer list can be instrumental to accurately valuing the business. Can you assemble the data quickly and can you rely upon it?
• Opportunity costs are real and significant. When your management team deploys a large part of its resources to solve data issues, such as struggling to derive pertinent facts from technical reports, those resources aren't available to focus on the immediate business decision. And in companies that have already downsized or outsourced their IT analysis resources, the problem becomes magnified.

The analysis of electronic data can often be a very time-intensive effort. The detail tasks connected with capturing, validating, and processing your data can take up 50 percent of the total business problem-solving effort. And the more time you spend here means less time for solving the business problem.

Even minor delays can have an aggravating effect on risks and achieving business objectives. Very often, the analysis solution requires you to create custom software. In most cases, an organization needs to validate the assertions it makes about the integrity of data assets and software tools. The investigation could focus on sales information located in a data warehouse, customer information in a legacy system, or the open item balance in an accounts receivable system.

A Level Beyond the Technical

A competent data analyst will give you good data. However, the best analyst will hand you the data and also tell you why it's right. This extra effort is a huge difference in today's world of business.

Now more than ever, a sense of security and confidence needs to be built in and assumed in the data that an organization's people are using. The team doing the work must work on a higher level than simply manipulating the system and its equipment. A first step would be to perform a series of diagnostics on the data to profile its composition. These tests provide valuable insight into the range, distribution, and continuity of key data points. For example, does the data contain repetitive occurrences of the same numerical value? Does the data contain records that are outside of the target analysis period? Do unanticipated gaps of days or weeks of activity exist? These tip-offs are just a few of the potential data problems.

The next step would be to compare data to other reference points, such as financial statements or transaction-control totals. Period-to-period comparisons can also be effective evaluation metrics. Building confidence in your deliverables is difficult without verifying your data with appropriate benchmarks.

The key to effectively using these techniques is to perform them at the start of the analysis process. This step lets you and the others you are working with know early on why data is suspect and how serious the problems are that you have to overcome.

Four Points of View

A key requirement is being able to substantiate how your data was created - every time and all the time. The approach I use is to assess the integrity of all the data before its use in the analysis effort. A time-tested methodology that I've found effective is to look at data from four different points of view:

• Relevance. When you're looking at information from a data warehouse, ask yourself: Is analyzing 100 million records cost effective? Clearly, you have to cull extraneous data. Identifying and discounting data that's irrelevant to the issue at hand is often a major challenge. Sort things out according to your criteria, such as geography, time frame, or the size of the transaction. Do this step early and you'll significantly help streamline the remaining tasks in the analysis effort.
• Reliability. How precise is the information? How well does the information tie to other audited information? For example, how solid and accurate is the linkage you're trying to establish between click-through volume and royalties? Why do records have missing data in required fields? Weak data entry controls and inconsistent data resource management can play a significant role in diminishing the reliability of corporate transaction data. Make sure that you confirm or qualify the reliability of the data you are working with before you move ahead with the remaining tasks.
• Veracity. Could your data have been tampered with? Was there an intention to defraud? The raw numerical information may be plausible, but does it accurately reflect the transaction that actually took place? Here, you must determine that truth exists. Auditing skills are pivotal to making these kinds of determinations.
• Privacy. In this step, you must consider several issues, including confidentiality and both U.S. and international laws. For example, the European Economic Community (EEC) is elevating privacy requirements. Is purchased information encroaching on someone's privacy? If accounts payable information is leaving the EEC, this act may violate the rights of an individual. In the U.S., particularly in the healthcare industry, privacy of patient information has become extremely important in view of the impending passage of the Health Insurance Portability and Accountability Act (HIPAA). You must address similar concerns about financial services industry data - particularly when it crosses state or national borders.

Presentation Issues

Data analysis professionals need a focused, prioritized, and goal-oriented approach to produce a good reporting deliverable. These traits can have a dramatic impact on both the overall effectiveness and efficiency of the analysis effort.

An important element in building effective analysis deliverables is shaping the presentation results. The goal is to make the deliverable relevant, informative, and easy to follow.

Rate This Article Rating: 1 (best) 2 3 4 5 (worst) Comments: Optional e-mail address:

I like a deliverable package to include five components, with the following logic, ingredients, and levels of detail:

• Summary. The objective of this section is to indicate the overall context of the deliverable to the reader. This section should include the scope of the analysis, any limitations, assumptions made, and the variables selected. It should be brief and concise - no more than two pages.
• Central observations and comments. This section outlines the specific objectives of the project and indicates the specific results and accomplishments in relation to each objective. This section, ideally, should be five to seven pages and serve as an executive summary.
• Summary reports. This section is usually in the form of tables, graphs, and charts to support and explain the analysis. It should be high level and focus on supporting the areas of significant observations and findings - a maximum of 10 to 15 displays.
• Detail reports. These reports include in-depth information needed to tell the story behind the data. This section is where the data volume is potentially high - try to control it. One technique is to provide only an excerpt and then cross-reference it to more expanded information provided under separate cover.
• Audit trail. Including this information assures the reader that the analyzed data has been verified and that it is reliable. Basically, this section describes what data you received, which data you used, and what data was set aside (and why) and clarifies any issues you uncovered that could affect the analysis results. A maximum of four or five pages is usually adequate.

Another technique to enhance the presentation integrity of your deliverables is to tag each page in the report with an identifier. This tag ideally notes both the version of the data used and the version of the report produced; this technique facilitates managing the risks associated with version control.

When all is said and done, data analysis tasks are on the critical path of the management decision-making process. Regardless of the task, your analysis team, equipped with some of the insights provided here, should be ready to streamline its approach to making the data tell the story as articulately and accurately as it can.

John B. Bloniarz (john.bloniarz@ey.com) heads Ernst & Young's Information Systems Assurance and Advisory Services Shared Analysis Center.