Crossposted from Accountability Lab
The Accountability Lab and Feedback Labs recently co-hosted an event at the OpenGov Hub in Washington, DC entitled “Impact: So What?” The idea was to generate an honest conversation around how we are measuring impact in the accountability and transparency space and what we can collectively learn from this process to help us improve. Preceded by a Twitter chat (#impactchat) and a twin event at the OpenGov Hub Kathmandu, the discussion brought together a diverse crowd of activists, non-profits, policymakers and donors.
Blair Glencorse of Accountability Lab facilitated the discussion, with panelists Dennis Whittle of Feedback Labs, Olive Moore of the Global Partnership for Social Accountability, and Olimar Maisonet-Guzman of IREX. Watch the video here. Key ideas could be summarized as:
- It’s all about people. Accountability is about giving people a voice and making sure that voice is heard. Jargon obfuscates the discussion, which is really about what people want and if they are they getting it. Ultimately, our greatest measure of success or failure is the perception of people on the ground. How do they judge our work and can we reduce the time and distance of our actions from those people we’re trying to help, so we can support them in overcoming their challenges?
- Impact isn’t always measured in numbers. Measuring the number of workshops or the number of people reached is not enough to tell an effective story about impact. We can still be too outputs focused. Often individual narratives of change can give deeper insight into what really matters. Coupling meaningful qualitative and quantitative data together can show long-term shifts in systems and mindsets, which are key to building accountability. How do we design measures that serve the purpose of citizens rather than our own log-frames and metrics?
- Accountability is political. This is an inherently political field and we as a community must determine what role we can play and the level of influence that is appropriate. This work is about helping citizens be active and engaged, rather than trying to change politics ourselves. Citizen engagement with power-holders can take the form of “sticks” or “carrots,” but recent evidence seems to demonstrate that a positive narrative around change can allow for movement beyond obstructionist politics.
- Scale does not equal impact. We need to think harder about how we connect the dots to understand the linkages between our own project level changes and larger system-level transformation. Maximizing scale is not necessarily achieved by replicating projects. Scale can mean many different things, including vertical or horizontal growth, greater geographic span, or deeper processes for accountability. There is no-one size fits all in terms of impact, but we have to get better at building coherence across our efforts to ensure our work is as effective as it can be.
- Structure may prevent change There is a tension between the “we know it when we see it” approach to impact measurement and the use of Theory of Change (ToC) type models. These tools help give clear direction and structure, gather data over time, and test assumptions, but they are the means not the end. How do we involve citizens in the development of our frameworks and ToCs? How do we keep these tools iterative, inclusive and open to adaptation, despite the risk of losing baseline measurements? These are issues that should be thought about at the design phase, not in the post-project review.
- Measuring impact is an adaptive learning process. M&E processes require formats that allow for impact learning and growth within projects from the outset. Accountability is not a linear process, and our efforts to measure it must bear this in mind. Progress requires doing, failing, learning, and adapting. It involves talking to our partners around the world about how we can do better. Organizations will be most effective where they allow for flexibility to follow impact as conditions change and refine and renegotiate their metrics over time.
- It starts with our own honesty. We need to match rhetoric with purposeful action on these principles in our daily work. This requires trying new ideas, continuously asking what’s working and what’s not, and iterating and improving accordingly. We need to be honest about what we can and cannot control, and to ask donors to give us flexibility to follow the impact. In order to build accountability around the world, we must start by demonstrating our own internal accountability.
We – along with a number of other organizations based at the OpenGov Hub, includjng Global Integrity, the Open Contracting Partnership and the Open Government Partnership – are keen to continue the conversation about learning and impact. If you’d like to be kept in the (learning) loop, please contact [email protected]
