Metrics are vital to managing a successful service. It’s how the service owner measures quality and performance: whether it’s compliance with a government requirement or regulation, customer satisfaction, or business process efficiency. It’s also how the service portfolio owner makes decisions around the retention or retirement of a service. These successes (and failures) must be quantitative.
Very often, unfortunately, a service is established without having ways to measure its success. The service owner gets more interested in the details of the implementation stages or tracking what the development team is doing, and they forget about the longevity of the service their product will provide and the processes behind it. The service owner may not consider ongoing management requirements, i.e. the metrics that are directly related to the usage of the service being deploy, outside of milestones reached. In most cases, a business process is developed without the consideration that it may need to be refined and improved upon in the future. (Sorry, PM, you probably haven’t created the perfect service or process on the first shot.) Service Level Agreements (SLAs) or other means of judging customer satisfaction are not always developed ahead of time, and Cost Management plans are sometimes limited to just “the budget” instead of ways to measure total cost, cost savings, or cost avoidance.
In a perfect world, a performance management plan would be created as early in the life of the service as possible. SMART (Specific, Measurable, Attainable, Realistic, Timely) metrics would be clearly defined in the planning phases of the service, taking into account the service’s development. While the service is in development or still relatively immature, the service owner would collect progress and compliance metrics. As the service matures, effectiveness of the outcome can then be measured, followed by the efficiency of the processes that produce and manage the solution.
As cliché as this sounds, one of the first steps in developing performance metrics is to identify and engage the stakeholders involved or responsible for the work because they are the most knowledgeable. These people include the developers of the solution, the existing or proposed customers, and those stakeholders involved in the management of a critical process that supports the service. Including the stakeholders from the beginning will ensure that everyone has a common understanding of what the metrics mean, how they are collected, and why they are important to decision making.
The service owner and stakeholders would then take a top down approach (similar to the below figure) to ensure the metrics align with the strategic goals of the service. Those metrics would be measured consistently, collecting them through a defined process, to allow continued service improvements throughout the life of the service.
But, this isn’t a perfect world. eMentum has gained a reputation for taking on and solving the tough challenges, and one of our teams was recently assigned to support a program supporting components as they develop technical applications as pilots for enterprise services. Each pilot is led by a different component, and each is managed by a different project manager, each of whom has a different approach to service management. We quickly learned that, in most cases, the pilots had not considered performance metrics. These metrics were not only needed to report progress to senior leadership, but also to judge the quality and success of the pilot as input to a decision whether the service could be leveraged by the entire Department.
First, the team met with the pilot project managers and the proposed customers of the service to determine the results desired and to align them with the customer requirements. Then, we identified their unique critical work processes to achieve these goals and determined how those processes could be measured. Examples:
- If the goals were technically driven, we identified ways of measuring the average processing times consistently achieved over the network.
- If the goals were centered around efficiency, we identified how their existing release management process could be used to call out enhancements and defects addressed on a monthly basis.
To establish targets, the team considered the standards which would act as a benchmark for the collected data. Throughout the development, we applied the SMART test when drafting the metrics to ensure their usefulness. After the detailed metrics were established (which were unique to each pilot), we reported them up to senior leadership to ensure the progress being made on the pilots was understood in the format requested.
The reality is that the situation eMentum encountered is not rare. Organizations often overlook the quality management aspects of projects, and ITIL processes are not fully engrained in our business cultures. The valuable project manager or service owner will be able to overcome those challenges and create metrics under circumstances that are not ideal, including after the planning phase is over and execution is in full swing.
Have you ever experienced a situation where a service was operating without metrics? How did you resolve the situation? We would love to hear your thoughts!
Thanks for reading! Do Good. Have Fun. Add Value.