One of the weaknesses in most approaches to Business Process Management has been their approach to process measurement. If you contrast the effort that most BPM practitioners focus on measurement and compare it with the efforts made by Six Sigma folks, it becomes a glaring difference. This is not to say that Six Sigma approaches are altogether satisfactory either. Six Sigma approaches tend to focus too narrowly on measures that track process improvement, and not enough on on-going measures of process performance.
Let's be clear about this distinction. In an ideal world, a management team ought to have defined the major processes that constitute the organization. Presumably this would start with one or more value chains, and proceed, by decomposition, to several major subprocesses. Further, the management team ought to have measures to define whether each of the major subprocesses is performing as they should. At this point we are not talking about defective processes – we are talking about the major subprocesses of the organization and discussing how we determine that those processes are satisfactory on a day-by-day basis. In essence, we are talking about a group of measures, which, if taken together, make up the Key Performance Indicators (KPIs) that the organization uses to determine if the organization is performing as it should.
Separate from these measures of ongoing process performance, we sometimes find that we need to upgrade or improve specific processes. We determine, for example, that a competitor is providing customer service better and cheaper than we are able to, and we launch a redesign project to improve our customer service process. As we explore the process to be improved we identify specific process measures that we will want to track – things like how much it costs to do specific tasks and how many units are produced by different groups in an hour. Some of these measures may be similar to the KPIs that we normally track, but most of them will be much more specific and more difficult to track. We track these specific measures while we are working on process redesign, and then tend to stop tracking many of them once we decide the process is now performing as it should.
So, we have two types of process measures:
- Ongoing Process Performance Measures – KPIs – usually measures of process outputs of value, and
- Internal Measures of Process Improvement – normally used when we are trying to analyze and improve a specific process
This Column will focus on the Ongoing Process Performance Measures or KPIs that every organization ought to track.
One of the key features associated with the Ongoing Process Performance Measures is their relationship to 1) what customers value, and 2) organization strategies. Indeed, one of the key problems in defining these measures is determining who you are trying to satisfy. Many BPM approaches (Outside-In, for example) seem to assume that the ongoing, external measures ought to be measures of customer satisfaction. The assumption here is that value chains and major business subprocesses are designed to provide value for customers and that a description of the value to be provided is a sufficient way to measure the success or failure of the process.
Obviously no BPM practitioner would deny that customer satisfaction is very important, but we need to qualify that concern with some other practical considerations. If one has, for example, worked on actual improvement projects, one is aware that concerns about costs, compliance, and profits are often the concerns that management emphasizes most. In other situations, as when management is seeking to implement a new strategic initiative – say the instillation of ERP throughout the organization in order to standardize the organization's financial table of accounts, to provide up-to-date information on case flows, or to comply with a major new regulatory requirement, as for example a law requiring a new type of corporate reporting – management will place significant emphasis on whether or not major processes are supporting and complying with the strategic initiatives of the organization.
At this point, some BPM practitioners might suggest that one look at the various Balanced Scorecard or Strategy Mapping approaches that have been popular in recent years. I initially tried to use this approach and linked efforts to define and track Ongoing Process Performance Measures to Strategy Maps. In practice, this proved less than satisfactory. For all their talk of emphasizing process, most of those engaged in the Balanced Scorecard and Strategy Mapping efforts are not really process oriented. Their definition of “process” does not match the use of the term in BPM and their actual practices are too focused on defining departmental-based KPIs.
A Stakeholder Diagram
The place to begin any serious analysis of a truly process-based KPI system is with an analysis of each given process. To do this, I advocate the use of a Stakeholder Diagram. In essence, we place the value chain or any major subprocess in the center, and then ask who the stakeholders are – who, in other words, is concerned with the success or failure of the process we are focused on. Figure 1 illustrates a simple Stakeholder Diagram for a Deliver Packages a major process used by a Package Delivery Company.
Note to begin with that this process is a little unusual in the sense that it has two major customers, one that sends a package and another that receives a package. In any case, both of those customers have traditional customer concerns. They want services on time, without problems, and at a reasonable cost.
Just as every major process has customers as a stakeholder, it also has what we have termed Company Management as a stakeholder. Company Management holds the process responsible for compliance to policies, for financial reports and high ROI, and perhaps for meeting the goals of strategic initiatives. Although we don't do it here, you could imagine a more detailed diagram in which we pictured the stakeholders of the Company Management unit. Shareholders, Banks, the Public, Government Agencies and others might all interface with Company Management. Another way of saying the same thing is that Company Management incorporates subprocesses that report financial information to tax agencies and to banks and shareholders. These management processes depend on the inputs Company Management gets from the Deliver Packages process.
The key to deciding whether a given process has a relationship with a specific stakeholder is to ask whether or not there are internal activities within the process that interface with the specific stakeholder. The Deliver Packages process has activities that provide information to Company Management. It doesn't have specific processes that report to Shareholders. Deliver Packages generates outputs to Company Management and Company Management consolidates the information and provides information directly to Shareholders.
Value Chains and other high-level processes may have many stakeholders but they always have Customers and they always have Company Management stakeholders who are very concerned with the value produced by the process.
A service process – which provides IT services – is also a major stakeholder of the Deliver Packages value chain. This process manages the development and the execution of software applications and databases that provide IT services to the Deliver Packages value chain. Note in passing, that if we shifted our focus, and focused on the Provide IT Services process, it would have several customers, one of which would be the Deliver Packages process.
A Stakeholder Scorecard
Once one has a Stakeholder Diagram and has identified all of the stakeholders of a process and has a general idea of what each stakeholder expects from the process, one is ready to move on and create a Stakeholder Scorecard. (See Figure 2.)
In essence, rather than being concerned with “organizational layers” as Balanced Scorecard practitioners are, we focus on what the process will provide for each of its stakeholders. We usually begin our list with the customers and proceed to Company Management and then to other stakeholders.
The Stakeholder Measures Scorecard can be a worksheet as shown in Figure 2, or it can be a matrix generated by a BPMS software package. The important thing is that every manager can access the Scorecard and knows how each process is to evaluated. In essence, a process is judged by its ability to satisfy ALL of its stakeholders. A process can fail if it doesn't provide products and services to customers, but it can just as well fail, along with the company, if it doesn't provide information to Company Management that is required by Regulatory Agencies, or return profits sufficient to satisfy shareholders.
The Stakeholder Scorecard shown in Figure 2 provides the minimum information we might want. It names each stakeholder, indicates what the stakeholder expects from the process, and defines a KPI.
Indicators vs. Measures
There are a lot of terms that are thrown around rather loosely when people talk about metrics and performance measurement. One distinction that is important and goes by many names is the difference between Indicators, which are general statements of something we want to achieve, and Measures, that define exactly what we want to achieve. Sometimes Indicators are called goals, or Key Performance Indicators (KPIs). Similarly, Measures are often called objectives.
Let's define a few more terms, and then we will be able to discuss this with more precision.
Vector. Indicates a trend, as for example, Increase, Decrease, Maintain.
Unit of Measure. The outcome we are focused on, as for example, Profits, Sales, Returns, Rejects.
Target. The way to measure the outcome. E.g. 50, $350, 6%, no rejects.
Timeframe. When we want to achieve our target. E.g. by 5pm, at the end of the quarter, next year.
Now, using these terms, let's be a little more specific:
Indicator (goal, KPI) A Vector + a Unit of Measure. E.g. Increase profits, Decrease Rejects.
Measure (objective) A vector + a Unit of Measure + a target + a timeframe. E.g. Increase profits by 5% during the next quarter. E.g. Decrease rejects to below 5 per month.
When we develop our initial stakeholder diagram we describe our Unit of Measure, and often include a Vector. When we complete our Stakeholder-Scorecard we use Indicators or KPIs and thus we list vectors and focus. Later, when we get more specific about exactly what we will do to evaluate if our Indicators or KPIs are met, we define Measures.
In some situations we go a bit further and describe the Data Source — where the raw numbers of documented events or interview notes will come from and exactly when and how we will collect the data.
If we add this additional information to our Stakeholder Scorecard, our column, labeled “How We Will Measure Success” in Figure 2 will end up subdivided between Indicators (or KPIs) and Measures which will define the specific targets and data sources. (This becomes especially important if one is doing this with a BPMS tool and that tool is going to provide measurement information to managers using the finished BPMS application as a way of monitoring the ongoing performance of a process.
This brief summary of process performance measurement has been just that – brief. Still, when compared with discussions that make it difficult to understand just how processes relate to organization strategies, or articles that suggest that process measurement is all about value created for customers, it provides a much more satisfactory scope. It starts with the process and uses the stakeholder concept to derive a specific set of metrics that can be used to measure value created by a given process for customers, company management and for other groups that also have an interest in the given process.
We have omitted a lot, including a discussion of how Stakeholder Scorecards can be arranged in hierarchies to reflect process hierarchies, or how Stakeholder Scorecards based on processes can be reconciled with Departmental measurement approach like Balanced Scorecard. We have also omitted a discussion of process improvement metrics and how we use measures to determine that specific changes in processes result in the improvements we desire. These are all important topics that any process practitioner will want to master. Suffice to say that this can be done and is done by using the same basic concepts we have described in this Column, supplemented by a few other measurement concepts. The key to all this, however, is to start with the specific process and work from there.