Tips for Avoiding Common Metrics Challenges
Many organizations spend a lot of resources to collect and report a number of metrics without focusing on the proper utilization of them or tracking the return on investment in the effort. In some cases it may be difficult even to find meaningful metrics.
Missing the right metrics poses a great risk to business decision-making processes. Decisions made in the absence of data or with the wrong data can result in cost overruns, schedule slippages, quality issues, dissatisfied customers and canceled projects. With the right metrics and an efficient analysis framework, however, companies can baseline current performance, set performance targets, identify opportunities for improvement, and enable course correction to meet goals and objectives.
But, as is true for any initiative, metrics programs face some unique challenges that need to be tackled successfully in order to derive real benefits. Following are some of those potential roadblocks, as well as recommendations that metrics teams can use to avoid such problems.
Lack of Metric Alignment
Any metric that cannot be tied to an organization’s goals and objectives needs to be discarded, unless there is a compelling reason, such as a specific request from a customer, for that metric. After this evaluation, a team may find that it lacks metrics that link to the organization’s goals and objectives, in which case, the team must decide on the right metrics. If the organization’s goals and objectives cannot be quantified, they should be reassessed; otherwise, the team will not be able to track progress toward the objectives.
- Evaluate and finalize quantifiable goals and objectives.
- Align existing metrics to the organization’s goals and objectives.
- Reassess existing metrics that do not match up to the organization’s goals and objectives.
Too Many Metrics and Reports
Many organizations generate a lot of metrics and reports that nobody uses. Sometimes, it is even difficult to track who is actually the owner for what is being reported. All reported metrics should be critical to the business, not simply ones suggested by best practices or textbooks. Efforts should be made to identify a small and balanced set of metrics that meets the organization’s needs, and are cost effective to maintain and run. All existing metrics should be assessed with key stakeholders to understand the relevance and importance of each. Redundant metrics should be discarded.
- Assess existing metrics and reports.
- Assess the cost of running a metrics program.
- Assess the need and associated cost for existing metrics and reports with key stakeholders.
- Discontinue redundant metrics and reports to reduce costs.
Focusing on People Rather Than Processes
In some cases metrics are used primarily to assess people performance, rather than process performance. This may lead to data manipulation or underreporting of metrics. If people’s careers are dependent on reported metrics, there is a tendency to hide facts or report incorrect data. Individual performance is important, but the focus of an initiative should be on process performance. This will lead to much wider participation within the organization, and when processes improve in performance, everybody wins.
- Emphasize performance and continuous improvement rather than individual performance.
- Encourage people to suggest improvement ideas and participate in efforts.
- Reward people and teams for improvements on performance.
Many organizations set targets without any thought to current performance, process stability or process capability. Industry benchmarks are helpful, but before applying these benchmarks to an organization, the team should analyze current performance to ensure that unrealistic targets are not set. Unrealistic targets create resistance within an organization and impact team and people performance. In some cases, they also lead to data manipulation or incorrect reporting.
Before setting any targets, the metrics team should ensure that processes are stable and that process capability can be measured in a reliable manner. Process capability should be measured from the customer’s perspective. If teams do not consult the customers, they may find that clients are still unhappy even when targets are consistently met. Involving customers at each stage of target setting helps teams set realistic and achievable targets that will meet customer’s expectations.
- Assess process stability.
- Kick-start improvement efforts to stabilize processes.
- Baseline process performance for stable processes.
- Set realistic targets based on current performance and industry benchmarks.
No Metrics Analysis Framework
Companies that generate metrics reports also should have a robust metrics analysis framework. Without a statistical framework for measuring, analyzing and improving performance, the value of metrics diminishes. Metrics and reports are a means to an end, not the end themselves. They should be analyzed to assess process stability and capability, and to trigger improvements efforts.
Teams also should ensure that reported metrics data is accurate. Verification and validation of data is of paramount importance as incorrect data will lead to incorrect conclusions. A robust metrics analysis framework will enable identification of root causes if processes do not go as planned and data-based corrective actions are needed. It also will help keep team members from overreacting or tampering with efficient processes.
- Set up a metrics analysis framework.
- Set up a framework for data validation and verification.
- Set up an improvement framework if processes are not performing to their potential.
Abhash Chandra is a senior Master Black Belt and information technology process transformation consultant at Tata Consultancy Services Ltd. in Mumbai, India. Chandra has assisted major corporations in deploying Six Sigma and has executed various Six Sigma projects and mentored others in their projects. Contact Abhash Chandra at abhash.chandra (at) tcs.com.