Information Impact: Business Analytics Renewed, Part 3

Creating decision value graphs is the final step in linking action to outcome

STRATEGIES FOR CHANGE

When you've understood your decision context, you still need to figure out which information in which situations is most relied upon for the decision made. (I discussed this process in Part 1.) And you need to figure out what combinations of improvement or degradation to which attributes of your information would improve or degrade your decisions by how much. These are your strategies.

It is important to recognize that the value of the information is not a function of the information itself, but rather of measurable attributes of the information. The concept of assigning value to information is a loose proxy for assigning value to the existence, or coverage (over some area), of information. Saying that a sales forecast is valuable is equivalent to saying that the existence of a sales forecast is valuable.

In addition to coverage, the other important measurable information attributes (again, as described in Part 1), are accuracy and timeliness (or speed). The value of information is thus the incremental decision value owing to an incremental improvement or degradation in decision quality resulting from the incremental improvement or degradation of some combination of information attributes.

In trying to reconstruct the full decision context, you need to tease out as much decision quality - information attribute linkage information - as possible. For example, you need to ask questions of the form, "What kind of attribute improvements to which information would have allowed me to make a one- or two-unit incremental improvement in my decision quality?" There might be several responses indicating that either an improvement of the coverage of some piece of information or an improvement to the accuracy of that same or another piece of information would have been sufficient to improve the quality of the decision made by one unit. You also want to find out which information attribute degradations might cause the quality of your decision making to slip one or more units.

Now you've come full circle relative to the first installment of this series. You've figured out the most important decisions that are regularly being made, you've built decision value graphs that capture the incremental value associated with incremental changes in decision quality, and you've associated improvements and degradations in decision quality with improvements and degradations in the quality of information attributes. All that remains is to associate costs with the changes in information quality attributes for you to be able to calculate the value of information and the return on information investments.

For the example in Figure 7, I'll describe where I left off in Part 1. Whereas with rough weighting you could figure out which information connects to which decisions, which information and which decisions were most important, and the rough pairing of information quality with decision value, now you can begin to evaluate the relative merits of trying to improve different information attributes in terms of the incremental value attributable to the improved decision making they support.

You will notice that some strategies result in lower decision quality. That's to be expected; changing priorities within a corporation may dictate the need for incremental divestments in some areas because they are more than offset by the increased returns elsewhere. And if it's a question of financial health in general such that less money must be spent on IT, it's important to know which cuts will produce the least harm.

You should also notice that the grid in Figure 7 is fairly sparse. This sparseness is realistic because the grid reflects the known, implementable information quality changing strategies. There are many reasons why different information attributes are more or less easily changed including financial cost, political cost, practical possibility, and the knowledge of how to effect change.

Finally, you should notice that the grid is filled with estimated values as opposed to the binned ratings of the first installment (which were reflective of the degree of precision in the various estimates). The reason is that each grid value represents either an actual or potential state of affairs that can be described as a strategy or project, such as by saying, "We estimate that by spending $50,000 +/-20% we can improve the accuracy of our long-term forecasts from +/-30% to +/-20%, which will improve our decision quality one unit, which translates into a decision value of $100,000 +/-25%." Therefore, the spending options and decision value improvements aren't continuous in any sense, but are quantified in a realistic but low-cardinality way.

At this point, you can use whatever financial methods you feel comfortable with to put a finished ROI-style value on each of your change strategies.

VALUATION FROM THE BOTTOM UP

In addition to the two top-down methods that you've seen in this series, you can try to obtain more precise estimates of information value by following any of two major types of bottom-up methods. Bottom-up methods leverage existing and creatable data in order to attempt to quantify all the linkages.

Historical analysis uses all available historical data, in addition to interviews with key decision makers, to improve the quantitative assessment of the value impact of decisions, the weighting of information used, and the information attribute impact on decision value tables. You can use historical analysis to create all three products of decision reconstruction. For example, in order to reconstruct the set of potential decisions and their associated values, you may look to the historical record to see what decisions with what results were made in the past or in other locales.

Rate This Article Rating: 1 (best) 2 3 4 5 (worst) Comments: Optional e-mail address:

Active analysis uses experimental design techniques to actively quantify all of the linkages I have described in this series. For example, if a retail organization has a large number of outlets (taking into account local variations), you could design an experiment to test the value of long-term sales forecasts in purchasing decisions by intentionally withholding that information from a certain group of buyers and measuring the results.

The choice of methods is not an either/or. I would typically recommend a breadth-first approach, then drill down where necessary. Self knowledge is an ongoing process.

Erik Thomsen [erik@dimsys.com] was cofounder of Power Thinking Tools, which developed the first OLAP engine with integrated statistics, visualization, text processing, and object management. He is a researcher and consultant for Dimensional Systems and focuses on integrated multitechnology analytic solutions. He is author of OLAP Solutions (John Wiley & Sons, 1997) and coauthor of Microsoft OLAP Solutions (John Wiley & Sons, 1999).

ON MULTIDIMENSIONAL DECISIONS

Plenty of decisions are multidimensional from both decision-taken and resulting-value perspectives. For example, a manufacturing process decision might involve a multidimensional selection of temperature, pressure, and time, and it might result in a multidimensional set of hardness, smoothness, and impurity coefficients. Nonetheless, at some point that manufactured product will be sold (or discarded) - thus affecting some primary organizational value. And there will be some mapping of result coefficients to organizational value and decision variables to output coefficients such that weightings could be assigned to the decision variables and the output coefficients. This mapping makes it possible to define unit changes to decision making and unit changes to decision value. The major difference relative to the naturally one-dimensional decisions is that there may be several decision options that are one unit better or worse in terms of their resulting value.