Author: Premal Desai
What is the meaning of Metrics
IT metrics are quantitative assessments that enable organizations to understand the performance of their IT initiatives.
Key Performance Indicators (KPIs) are a subset of metrics to demonstrate/explain how effectively specific business goals linked to objectives via Projects are achieved. [Projects are enablers of change & every project should have Goals and Objectives]
Example of Metrics - Business performance metrics, Sales performance metrics, Project management performance metrics, Employee performance metrics
Difference between SLAs and KPIs
SLAs establish the baseline performance expectations and monitor the agreement and what’s done to meet the expectations as per written or formal contract and communication.
Conversely, KPIs report on the efficiency or effectiveness or success in satisfying expectations or achieving organizational goals and objectives.
SLAs and KPIs have unique specific purposes. SLAs ensure performance metrics stay above a specific level of success. SLAs are more operational in nature while KPIs, however, promote optimal performance and help ensure improvements occur to deliver the expected results.
Examples of SLAs – Availability, Error rate, Resolution time, Service provider response time, Turnaround time
Some examples of KPIs include – Average customer lifetime value, Customer retention, Debt versus equity ratio, Employee turnover, Revenue by market, Sales qualified leads, Year-over-year growth
So one can summarizing in stating that – SLAs are sub-set to KPIs.
Why Metrics are needed or collected
Following are some of the benefits of collecting Metrics and analysing it –
- Metrics drive your operating model/BAU & enables improvements
- Gives clarity for performance expectations
- Metrics helps to manage activities more objectively
- Metrics helps to drive business execution
- Enables to decide or focus on ‘what is important’
- Establishes objective accountability
- Metrics Make the Processes Objective – Effectiveness and Efficiency
Different Agile and Scrum Metrics
Two most common Metrics –
Sprint Burn-down – The sprint burndown chart enables to visualize how many story points have been completed during the sprint and how many remain, and helps forecast if the sprint scope will be completed on time
Agile Velocity – Velocity measures how many story points were completed by a team, on average, over the past few sprints. It can be used to predict the team’s output in the upcoming sprints, helps in Sprint planning
Others Metrics –
Lead Time – Time taken to convert story to actual work/development completion or can be extended to measure realization of requirements/need to release
Cycle Time – Time taken to convert user story to ‘DoD/Definition of Done’ state
Code Coverage – Linked to Unit Testing
Release Net Promoter Score – Net Promoter Score (NPS), calculated for a software release, measures whether users would recommend the software to others, do nothing, or recommend against using it. It is an important gauge of customer satisfaction.
Cumulative Flow – This is a kanban metric which shows the status of tasks – if large number of items or number under the specified task – indicates a problem or bottleneck and needs attention. Cumulative flow lets you catch problems in mid-process before they result in delayed delivery
Other Kanban Metrics are – Blocked time, Control Chart, Work Item Age
Failed Deployment – Measures the number of deployments V/S Failed Deployments – Helps to determine quality of environment and helps in determining whether teams are really building potentially shippable software
Escaped Defects / Post Release Incidents – The number of bugs discovered only after a build or release enters production [Termed as Defect Leakage also]
Different SAFe Metrics
Return on Investment, Value for money, Value streams are some of the important parameters which business stake holders looks. To help businesses find answers to these questions, SAFe 5.1 pays attention to three core metrics.
- Outcomes: Outcomes will let you know whether your solutions meet the needs of your customers and even your business.
- Competency: Are you well-aware of the practices that deliver customer value?
- Flow: How efficiently your business delivers value to the customers.
SAFe is a flow-based system. In turn, each of the metrics suggested by Flow Framework applies to SAFe. SAFe also explains Flow Predictability as a concept. The goal of this concept is to evaluate how Solution Trains, ARTs and teams deliver against the planned goals thus enabling business to achieve ROI, Value for money and eliminate DOWNTIME/Waste. [Lean IT concept]
As indicated above, Metrics in form of KPIs or SLAs are important for every project or service or activities performed and hence need to be collected and analysed. Agile comes with the promise of a higher quality product, a more dynamic team, and more satisfied customers — and agile metrics can provide the proof. Initially start with few metrics collection and analysis and add it at later stages of attending maturity. You will start to see the benefits of your efforts represented in a tangible way.