By Jonathan D. Breul, NAPA Fellow
I’m always intrigued by how other countries have tackled a management challenge that we’ve been grappling with for years here in the US.
For example, the US federal government has developed a solid technical framework for measuring and reporting on the performance of its programs and selected outcomes over the past five years. However, it has yet to develop an overarching analytic framework for assessing broader national progress.
There was a proposal championed by the Government Accountability Office in the late 2000s to create a set of key national indicators. It even issued a report on how a number of other countries had developed such systems. But little is done by the federal government, other than a table of 96 social indicators in the annual budget. Last year, former MicroSoft CEO Steve Ballmer established a nonprofit to create an on-line set of national indicators, dubbed USAFacts. It provides public information in a number of policy areas, such as small business or health, but not necessarily comparative benchmarks with other countries. Its data tends to be aggregated at the national level, not geographic. However, it oftentimes provides trend data, so you can see changes over time in a particular metric, such as mortgage interest rates or the number of patents issued.
So, what are other countries doing on this front?
Background
I had the opportunity to sit in on a presentation at the World Bank by a visiting delegation from Saudi Arabia, here to tout their progress in measuring their country’s progress against a national vision for where they hoped their country would be by 2030. Interestingly, they’ve made progress in several areas of public management where the US has talked about, but not yet acted. Interestingly, their progress extends what I saw when I visited Saudi Arabia back in 2009 and learned that their Institute of Public Administration had put in place a five-year program to develop performance measures for their government and train employees on their use.
Creation of a New Performance Unit
But in 2016, the country’s new leadership released a 15-year visionary plan – Saudi Vision 2030 – that committed the country to becoming a “global investment powerhouse” and reduce its dependence on oil. To track progress toward the goals in this strategic plan, a newly-formed independent agency was put in charge – the National Center for Performance Measurement (which goes by its Arabic term, “Adaa”). And unlike its predecessors, this performance measurement and management unit reports to the Prime Minister and has a governing board chaired by the Crown Prince. As a result, government agencies pay attention!
Adaa differs from other traditional government agencies in the Saudi government in that its 140-person staff was recruited separately from the regular civil service and reflects the ethos of a private sector operation – somewhat like the operation of our US Digital Service.
Its key roles are to:
- Track progress towards the Saudi Vision 2030 goals. This is being done via measures and analytics that are reported to the Prime Minister and Council of Ministers via dashboards and quarterly reports for each public entity.
- Strengthen agencies’ capacity for conducting performance measurement. It is training government employees on the basics of performance measurement and management. They’ve trained about 4,000 so far and have created an “ambassador’s program” of about 90 train-the-trainers. They plan to create an e-learning platform to reach 200,000 trainees by end of 2018 and one million trained public servants by 2010.
- Provide the public with information on how their government operates. It has created a progress portal that is currently only available internally within the government but the goal is to create a public-facing version in 2019.
According to Husameddin Al-Madani, the Director General overseeing Adaa, its initiatives have evolved rapidly and they are in the process of moving from a role where Adaa focuses on producing reports to a role where it is actively informing decision-making. To that end, as the director general of Adaa, he sits on both the Kingdom’s Strategic Management Committee and its Budget Committee.
Al-Madani says Adaa collects data at three levels:
- Outcome-based Key Performance Indicators: measuring the progress towards Vision 2030’s approved targets and objectives.
- Execution Level data: collecting data on milestone achievements of Vision 2030 realization projects and initiatives.
- Service-level data: measuring and collecting data on beneficiaries’ satisfaction with government services.
While Adaa largely focuses on Vision 2030 outcomes, it oversees a performance measurement framework where agencies manage their own operational data and are accountable for those elements via other, more traditional channels.
Progress to Date
Al-Madani says that Adaa did not have to start its performance system from scratch. It inherited an initial inventory of about 3,070 agency-level operational measures that had been developed by previous efforts. This allowed them to get a quick start at the execution level. He also says that they now have a set of metrics organized around the Key Performance Indicators that undergird the key elements in the Saudi Vision 2030 strategic plan.
In addition, Adaa has begun collecting metrics related to improving “beneficiary experiences” using techniques common in the US government – public satisfaction surveys, focus groups, “mystery shoppers,” and real-time data using the newly launched mobile app that enables beneficiaries to rate public services provided by more than 28 thousand public service centers across the country.
Al-Madani says that in addition to having developed a performance measurement framework, they have collected 4 quarters of performance reports so far and are in the process of conducting data-driven reviews, with a focus on outcomes, not just progress.
A Fledgling International Performance Hub
In addition to having developed an internal performance measurement system for the Saudi government, Adaa has developed an open source tool to “benchmark Saudi Arabia’s progress against other countries and global standards.” This International Performance Hub is in beta development and ultimately will allow international comparison across 12 pillars of socio-economic wellbeing (comprised of about 700 indicators for about 200 countries). The indicators are organized into thematic areas, such as education, health, and security and the data is extracted from international sources such as the World Bank and OECD.
Within each thematic area, or pillar, relevant country-level data is made available, along with mapping visualization. For example, under the “Education” pillar, there is a framework with categories for dozens of measures of student achievement, teachers, expenditures, etc. However, for many countries, the categories are currently blank. Data that should be readily available, such as the graduation rates from universities in the U.S. or Canada is unreported, but graduation rate for Kazakhstan is available for one year, 2017. However, for other pillars, such as “Economy,” there is more data. For example, the “number of days to start a business” is reported for most countries (for the US, it is 5.6 days!).
In addition to cross-country comparisons, you can create a dashboard for an individual country and create comparisons with other countries. The hope by Adaa is that this website will encourage countries to compare their international rankings and support data-driven decisions within their own country in policy areas of importance.
As the Saudi experience shows, there are ambitious initiatives underway that might serve as inspiration here in the US at the federal, state, or local levels.