A Key Performance Indicator (KPI) is a quantifiable measurement of an aspect of an organization’s operation that tracks its progress towards a mandate or objective. For public sector bodies, KPIs are key to managing organizational effectiveness and efficiency. They convey information. They help public sector organizations to make decisions, allocate resources and identify problems. KPIs bolster the accountability of an organization and instill confidence in stakeholders and staff.
Tracking of KPIs typically includes setting a target or benchmark for each individual indicator. The number produced over a quarter or year is then compared against the benchmark and previous periods. Deviations from previous results and current expectations should be assessed to understand why they have occurred and what the implications are. There should still be review of performance indicators that match expectations and maintain consistency. This is to confirm that the underlying operation that produced the forecasted number indeed moved substantively towards the desired regulatory objective.
Potential KPI Pitfalls
Even though KPIs are an important tool for measuring an organization’s performance towards its objectives, they must be meaningful. The underlying factors that went into the numbers produced must also be scrutinized and understood. Otherwise, KPIs could cause people to conflate the performance indicators with organizational objectives. Inadvertently, this could result in undesirable regulatory outcomes. KPIs are a means to an end, not an end in itself.
Too Many KPIs
In setting targets and benchmarks, it is important to keep in mind not to over-KPI. Statistics should not be kept for the sake of keeping statistics. KPIs should be meaningful and be tracked to further their purpose: to help make decisions, allocate resources and identify problems. KPIs that are collected and reported for the sake of data collection divert focus from critical aspects of the organization’s operation and what it needs to achieve.
In addition to diverting focus and precious time and other resources to tracking, too many KPIs are also confusing to the public and other stakeholders. KPIs should relate to something important, not tangential to an organization’s function and accountability.
KPI Misconceptions
In rethinking a KPI strategy, it is important to distinguish between targets and measurements. Organizations want to achieve or exceed some of the forecasts or benchmarks set for KPIs. For other KPIs, hitting the benchmark is not truly beneficial for the organization and is not consistent with its objectives. These benchmarks are not targets per se. They are simply statistics that help to inform the organization how to properly allocate resources.
Bigger is not always better. It is not always preferable to surpass targets for inspections, investigations and other key important operational functions. Sometimes it is. Sometimes it isn’t. What is important, is to stay focused on the value brought by a particular KPI. It is critical to understand what led to each performance outcome and why shifts occurred from previous periods and expectations. It is necessary to get to the reasons why a particular number has been produced.
Rectifying KPIs - Examples
Public sector bodies that issue licenses often track the number of licenses revoked. Revocations result from unlawful and dangerous activity by license holders and other conduct that is contrary to the public interest. Public sector organizations forecast how many license revocations they expect because sufficient resources must be devoted to administering the revocations and upholding them if challenged on appeal. Although the organization forecasts the number of revocations, this ought not to be considered a target or performance indicator. For any forward-thinking regulator, the theoretical objective is to achieve complete compliance and conduct in accordance with set standards. Trying to achieve a minimum number of revocations is not a strategy. Regulators want to work with their stakeholders so that conduct leading to revocations is eliminated. When these statistics are mistakenly thought of as targets, organizations are at risk for misdirecting strategy that runs counter to the public interest.
One number alone cannot paint a complete picture of strategy, even if that number is chosen carefully. A number can partially embody an organization's objective but that number cannot be the objective itself. Using inspection functions as an example, an increase in the number of inspections (with the same number of inspectors) could indicate that inspectors have become more efficient and effective with their time. It could also indicate that strategies and processes have been implemented to facilitate more inspections through reduced travel time or scheduling efficiencies.
Alternatively, an increase in the number of inspections could indicate that inspections have become more superficial to guarantee that targets are hit. It is not enough to reach or exceed a target. The underlying factors that went into the performance result need to be understood. In fact, a decrease in the number of inspections could be reflective of successful strategies implemented for modern approaches to responsible regulation. Organizations that adopt risk-based approaches to inspection frequency should experience an increase in the amount of time it takes to complete an inspection. Put simply, it takes more time to inspect a company that perpetually shows up with compliance issues. It takes less time to inspect a company that is in perfect compliance. Regulators ought to focus on the places where they can find and address compliance issues, not the easier inspections that can be done in less time to ensure that a target is hit.
A critical assessment should similarly take place when reviewing offence statistics. Like the example above, an increase in number of charges (with the same number of investigators) could indicate that investigators are focusing their efforts and are being more efficient and effective with their time. Alternatively, it could indicate that charges are being laid without merit or investigations are being closed prematurely, with incomplete collection of evidence. This can occur when investigators feel pressured to move on to the next case when individual production statistics are measured against peers. Of course, not all investigations are equally complicated. An increase in the number of charges could indicate that investigators are pursuing low hanging fruit, violations that are less critical to an organization’s mandate while avoiding investigations that involve more serious matters but are more time intensive to pursue. Again, responsible and accountable regulation means getting to the truth behind the numbers.
Moreover, there are consequences to increases in charges. Charges have to be prosecuted. This aspect of the organization’s operation must be appropriately resourced. An increase in number of charges based on more efficient and effective investigations is positive for an organization, but not if the organization fails to allocate proportionate resources to the resulting prosecutions.
These issues surrounding performance indicators are applicable to all aspects of an organization’s operation, especially for lawyers in the public sector who are sometimes called upon to navigate parallel proceedings on a file. One common KPI for public sector legal departments is how long it takes to complete a proceeding. But it is important to recognize that speed is not always desirable. Where there are parallel regulatory and criminal proceedings on a file, the regulatory process can often be completed faster. Unfortunately, a faster regulatory process could cause interference with a police investigation or criminal trial. Often there is value in letting the criminal investigation and proceeding wind its way through the court system before the regulator takes action, particularly in cases where the criminal matter has already advanced before the regulator’s involvement. An organizational KPI that strives to limit investigations and regulatory proceedings to strict time limits could consequently be counterproductive to the organization’s mandate and objectives. The many positive aspects to maintaining KPIs must be insulated from situations in which it is best to deviate from the ostensible objective of a performance measure.
These issues can be reconciled to arrive at meaningful, informative and constructive KPIs. It is important to keep in mind that KPIs are simply a tool to help decision making, resource allocation, problem identification and build accountability. By taking a sophisticated and thoughtful approach to KPIs, public sector bodies can achieve these purposes without the KPIs themselves inadvertently causing undesired outcomes.
Any article or other information or content expressed or made available in this Section is that of the respective author(s) and not of the OBA.