The Measure of Success

Apply metrics to bring out the business in e-business initiatives

Two months have passed since your latest e-business project was "moved into production." Your production support team has fixed a couple of minor bugs identified by the users, but nothing serious has been reported. You sit back in your chair, pat yourself on the back, and congratulate yourself on another successful project implementation.

Not so fast. Do you know exactly when and how often your users are using your Web application? Do you have an accurate assessment of the application's influence on your supporting business processes? Can you confirm that the business objectives that drove this project have been met? Can you show your business sponsors that they are receiving a return on the investment in your project? Unless you've implemented a business process metrics program, I doubt it.

CONFIRMING SUCCESS

As the Gartner Group stated so succinctly, "E-business is about the 'business,' not the e." In addition, any e-business project worth its salt needs a business process metrics program to prove that the business objectives that drove the "e" project in the first place have been met. Historically, the success of business solutions hasn't been measured using business process metrics, and their business sponsors have also deemed most of these projects, from 50 to 80 percent according to the Gartner Group, unsuccessful. Clearly it's time for change.

Metrics are needed to gauge the success of any newly implemented business solution - technology or otherwise - and perhaps more important, the effect of this technology on the business and its processes. You have many ways to use the information the metrics provide. For example, it can help you identify areas that need improvement so you can streamline the business process and generate requirements for future iterations of the software application.

As the business process designer for The Dominion of Canada General Insurance Co., I recently led an effort to implement just such a business process metrics program. Our e-business strategy is to provide Web access to our policy and billing information for our business partner - the independent broker. Our metrics program is helping us determine if we have met our goal to create an environment that promotes ease of doing business. Here's how we did it.

AGREEING ON THE METRICS

To determine the business process metrics for the project, we focused on confirming the meaning of the project objective as stated by the project sponsors. We researched existing documentation to create a list of objectives, which was then ranked in terms of importance by the project sponsor. As an example, one objective included on the list was "Meet the broker's need to answer billing questions from an insured (without contacting us, by using the Web)."

Using this list, we went through a rigorous process of brainstorming possible measures for each objective. We asked: What do we expect to change after our e-business project is implemented? What will be "better"? What difference in our business processes will we see? This exercise led us to create a list of 40 potential measures.

To narrow down our list, we researched and applied best practices in measurement. We determined that they should be direct (vs. indirect), automated (vs. manual), and available for measurement both before and after the project is implemented. This last criterion is critical to baselining and comparing effects of the business solution.

The measures must be tested to ensure they don't create an unwanted behavior. In other words, you must consider how the staff that the measurement affects could manipulate the process measurements. Finally, processes should be actionable by management and timely in the context of the business cycle.

To test our metrics, we created a series of checklists and tested each metric against these criteria. In the end, we had a list of 10 potential business process metrics.

One benefit we identified from the objective, "Meet the broker's need to answer billing questions from an insured (without contacting us, by using the Web)," was "fewer calls to the customer service department." From this benefit, we identified the metric, "Count the number of inquiries to customer service regarding billing questions."

OBTAINING THE COMMITMENT

Our next step was to gain management commitment - a crucial step for any project. We created a formal document to communicate and sell the need for this program. It summarized the program's objectives and outlined the process we followed to identify the potential metrics. A proposed implementation plan was also included to suggest how to collect, analyze, and report on the data. As the program would affect all our regional offices, we also included potential roles and responsibilities. We presented this report to the project sponsor and a senior management committee. It was well received and swiftly approved.

Next, we began to implement the metrics. At this point, we learned an important lesson. Did you notice that we began this program halfway through the development of our e-business project? This timing came back to haunt us. Because the metrics weren't identified early in the software development life cycle as part of the business requirements, we encountered a number of stumbling blocks in the design stage. We quickly learned that the design of the Web application wouldn't let us automate our proposed metrics without a lot of rework and performance loss to the Web application.

But we knew the importance of the metrics program, and therefore persevered with a hybrid solution. We identified measures obtainable from our existing legacy system and implemented temporary (although manual) data collection procedures in our field offices. We quickly learned that the critical success factor in this manual approach was a clear commitment from field managers to communicate the need for the project and support the ongoing collection of data as part of the daily workflow.

Rate This Article Rating: 1 (best) 2 3 4 5 (worst) Comments: Optional e-mail address:

It has been five months since we began measuring system use, calls, mail, faxes, and errors in our eight regional offices. We now have a database that lets us identify point-of-contact trends for the company as a whole and for each office in the field. With this baseline data, we have a much better understanding of the Web's potential effect on all stakeholders, our internal staff roles, and supporting business processes. Each group of the aforementioned audiences is now using this information.

As the Web application launches, we are confident that we're well equipped to provide an accurate assessment of the Web's effects on our supporting business processes. We'll know exactly when and how often our brokers use our Web application. We'll use the data collected, both before and after the Web is implemented, to confirm that we've met the business objectives that drove this project and identify areas for enhancement for future development stages of the project. And last but not least, we'll have the appropriate information available for our business sponsors to confirm that they are in fact receiving a return on the investment in our project. Will you?

JANETTE SIMPSON [simpsonjanette@hotmail.com] is a business process designer at The Dominion of Canada General Insurance Co. She recently completed an MBA specializing in MIS and marketing.