On 10 October 2014, at the World Bank/IMF Annual Meeting in Washington Integrity Action co-organized a session on the 100% feedback agenda with Tucker Severson from
and Jean-Louis Sarbib from developmentgateway.org/
When Jim Yong Kim addressed the World Bank/IMF annual meeting in October 2013 he outlined a vision of “One Group and Two Goals.” During his 2013 speech, Jim Kim highlighted three examples of indicators of improved performance by the World Bank that he would personally be following:
- To reduce transaction times by a third from conception of a project to first disbursement of funds.
- To become a better listener. For projects with clear beneficiaries, commit to get feedback from every single one of them, 100 percent (in 2012 the rate was 34 percent).
- To add rich detail to its maps so that anyone will be able to go online, click on the maps, and immediately learn where the Bank is working and what it is doing.
If the 100% feedback agenda is implemented well we think it has the potential to mark a real step-change for international development assistance. It could be truly transformative.
No development agency has committed to such a vision before. Intuitively, it may seem like an obvious case to make and some may lament that it is long overdue. The more important question now is not to look backwards. The challenging question is to unpack what it means to do feedback well.
Here are two insights we want to share and challenge the World Bank to think about:
First, it needs a key performance indicator (KPI) to assessing whether the 100% percent agenda is delivering value and that is not just being treated as a box-ticking, compliance exercise (which would be easy to get away with). A powerful KPI the World Bank can use is the fix-rate, the rate at which problems are resolved to the satisfaction of the key stakeholders. The Bank has recently published its Citizen Engagement Strategy which outlines a remarkably comprehensive strategy. They should be commended for the scope of their ambition. The fix-rate can be applied to most aspects of this strategy. The strategic framework paper for Citizen Engagement has not yet been published. It will be coming out shortly. But in the framework they distinguish between four kinds of engagement:
- Complaints: User-identified problems
- Monitoring: Users assess predefined performance goals
- Suggestions: User-identified methods to improve service delivery
- Satisfaction: Users assess quality of services provided
The fix-rate can definitely be used for the first two kinds of engagement and to some extent also for the third. Satisfaction is the easier of the four to measure.
A feedback system that uses the fix-rate as a KPI places the emphasis on being results oriented and delivering solutions, rather than simply identifying or reporting problems. As such, it will contribute to improving service delivery and development outcomes. A rising fix-rate will improve public trust in government and public office holders, improve the effectiveness of public services and assist policy makers in identifying the policy-level changes that both reduce the incidence of problems and further improve fix-rates.
The second insight is that a robust and resilient feedback system combines three types of feedback (see brief examples below).
- Type A: Principal-initiated and managed feedback system, reporting directly for example to a Presidential Delivery Unit, the Prime Minister’s Office, a Governor or Mayor. Type A feedback systems enable Principals, to get real-time feedback on problem hot-spots, the effectiveness of their departments and to initiate appropriate remedial actions.
- Type B: Manager-initiated and controlled feedback systems, for example for a specific health service, school district, or a major infrastructure development project.
- Type C: User-initiated and owned feedback systems actively engage citizens in sharing responsibility for resolving the problems that affect them. In the commercial world two well-known examples of user-initiated and owned feedback systems are http://www.tripadvisor.com and www.yelp.com; and services as http://www.ebay.com and https://www.airbnb.com couldn’t exist without user-generated feedback.
Type C is more complicated and the capacity to implement this in many countries is weak. But it’s the key to a robust approach. http://seeclickfix.com/, https://www.fixmystreet.com/, and http://www.developmentcheck.org/ are three powerful examples of this approach that focus on public services.
User-initiated and owned feedback systems (Type C) are important for three main reasons:
- Citizens are more likely to contribute their time and efforts to resolving problems when they have ownership of the tool and some part of the process. In many situations public office holders are hard-pressed to resolve identified problems on their own, often because they lack the capacities or resources to do so. When citizens become part of resolving the problems that affect them the onus of responsibility shifts from the adversarial politics of reporting problems and making public officials responsible for the resolution of problems to a greater sense of shared responsibility and an orientation towards finding lasting solutions. Widespread adoption of such an approach, including the engagement of youth, instils a culture of integrity, citizenship and shared responsibility. And such systems can be more responsive to local needs and priorities.
- Type A and B feedback by definition requires the buy-in of either a principal or a manager. They cannot be implemented without their approval. Type C user-owned and generated approaches can be initiated without prior buy-in or approval – that can come later. The most successful approaches of Type C are collaborative and not adversarial. So the strategy is ultimately to get the managers and/or principals on board. But it remains an important distinction that their prior consent and engagement is not a pre-requisite.
- Type B feedback mechanisms have a tendency to be driven by a project cycle. What happens to the data 2 years after the project is over? In most cases that data is unavailable to end-beneficiaries, other administrators or principals. The result is a loss of organizational memory and learning and any option of comparing one project’s effectiveness with another. Type C feedback, however, lives on and the data cannot be removed once a project is completed.
The combination of all three types of feedback will create some duplication and redundancy. This is intentional because a system with some built-in redundancy is more resilient, cannot be tampered with easily and is more likely to deliver transformative results.
Keystone Accountability is creating what is going to be biggest portal and ‘marketplace’ for feedback tools and approaches that they’re calling the Feedback Commons. It’s due to be launched early 2015 and it will include examples of Types A, B, and C.
The job descriptions and targets of the World Bank’s senior management were updated a few weeks ago. We have been told that the 100% feedback agenda was included in their personal performance targets. That is evidence that this issue isn’t going away and that the top management means business. The World Bank has committed itself to a bold vision.
Documents from the World Bank make it clear that management is committed to this agenda because the “right thing to do.” Now we need to muster the evidence and put forward an unequivocal case that it’s also “the smart thing to do,” as Jean-Louis Sarbib said during our session at the World Bank last week.
I just met with David Bonbright of Keystone Accountability (http://www.keystoneaccountability.org/), who has been implementing feedback systems for years. He’s a passionate advocate of their importance. He says “The goal of constituent voice systems is to create a responsive culture of soliciting and responding to feedback. We use the data to improve relationships. Eventually, it’s a performance management tool that can give you predictive indicators so that everyone can do a better job.” A few years from now we look forward to reviewing extensive evidence from the World Bank that this was one of their smartest investments.
Annex – Feedback system examples
Type A: Principal-Initiated and Managed Feedback Systems
- “Government Asks” is a multichannel (web, SMS and offline) mechanism to crowdsource policy solutions that has been running in the Brazilian state of Rio Grande do Sul. 360,000 citizens have voted on 36,000 policy proposals drafted by citizens themselves.
- The “Citizen Feedback Model” of Punjab province in Pakistan contacts citizens proactively (http://www.punjabmodel.gov.pk/) by SMS after they have used a public service to rate their satisfaction and check whether bribes were requested. The information is relayed to a unit that reports directly to the Chief Minister.
- “Hello Sarkar” (“Hello Government”) in Nepal is a reactive complaints mechanism (http://nepalofficers.blogspot.co.uk/2011/11/hello-sarkar-hello-government.html) using widely publicized hotline, SMS numbers and email so citizens can raise problems and complaints with government. The unit is located in the Prime Minister’s Office.
- The World Bank’s own Presidential Delivery Unit dashboard shares key performance information with all the World Bank’s stakeholders, making it another example of Type A feedback: http://pdu.worldbank.org/sites/pdu2/en/about/PDU.
Type B: Manager-initiated and Owned Feedback Systems
- The World Bank’s OnTrack platform supports SMS and online feedback loops between citizens, government, NGOs, implementing agencies and World Bank staff around World Bank-funded projects. It is currently being implemented in Bolivia in the Rural Alliances Project (PAR) and the Bolivia infrastructure programme Barrios de Verdad (PBCV).
- The Department for Rural Development in the State of Andhra Pradesh in India runs one of the biggest and most comprehensive social audit and feedback systems, with 21,000 villages that have been audited 6 times since 2006, With a staff of 1,300 they achieve a fix-rate of ca. 46%, uncovering on average 200-500 problems in each village social audit.
- Uganda’s U-Report SMS-based polling mechanism was developed by Unicef to get real-monitoring of social indicators. More than 200,000 U-Reporters have registered to take part across Uganda.
Type C: User-Initiated and Owned Feedback Systems
- CheckMySchool in the Philippines http://www.checkmyschool.org/ has delivered a consistent of 50-85% fix-rate for hundreds of identified problems across the school system (http://www.integrityaction.org/sites/www.integrityaction.org/files/documents/files/CTL_conf.pdf for a break-down of the data and more information and case studies).
- SeeClickFix (http://seeclickfix.com/) has contributed to resolving over 1 million problems identified by citizens across over 200 cities in USA and Canada.
- DevelopmentCheck (http://www.developmentcheck.org/) is a tracking and reporting tool for Integrity Action’s Community Integrity Building approach, which has improved services for over 4.5 million people in 11 countries. Its fix-rate is on average over 50%.
 For more information please see: http://www.integrityaction.org/statistics-measuring-fix