The quest for a killer KPI

Articles & Reports
 |  
Mar 2022
 |  
MIT Sloan
Save to favorites
Your item is now saved. It can take a few minutes to sync into your saved list.

What: The simplification of performance metrics can contribute to the business development, provided the considered metrics are the right ones.


Why it is important: Simplifying and having the right KPI has dramatically positive consequences on organisations and teams, and can be structurally transformative.


Agoda, an Asian-based travel company, discovered that it had too many goals and metrics, creating confusion and waste of energy and resources. It then decided to create a singe key performance indicator in order to unify behaviour between teams and be a shared currency across them.


They share their methodology in this MITSLoan article. This went through a series of A/B testing and test and learn processes on their front-end website, initially with one-off experiments, but soon they centralized learnings into a single unified system, and “experiment engine”. They also understood that the speed of the experiment engine had to be dramatically increased, created a new KPI on speed, and went from a dozen experiments per quarter to more than 1,000. However, this generated bugs, so a KPI constraint was introduced, in terms of code quality.


Soon, they realized that the velocity KPIs was not always focused on the conversion rate itself: teams were rewarded for testing ideas, not for testing ideas with an increased conversion in mind. This is why the primary KPI was replaced by the notion of “incremental bookings per day”. This led to a dramatic increase in terms of productivity, cooperation between teams and will to test new ideas. This also allowed to simplify teams and reduce the hierarchy as a sense of cooperation spread across the company.


MITSloan sums this approach through the following 9 steps methodology:


  1. At the beginning of each project, define a primary metric of set of metrics that will gauge success,
  2. Assign a KPI to every project
  3. Look for a KPI that is relevant across teams, to create a shared language
  4. Try to break that KPI repeatedly,
  5. Revise the KPI again and again
  6. Define a primary constraint that needs to be met while the company strives for the KPI
  7. Keep the data clean and reliable
  8. Accelerate innovation and experimentation, but limit errors
  9. Let the process of measurement inspire the corporate culture.


The quest for a killer KPI