When I met Aminata in a central Malian village, she asked me whether I was with the people with the yellow trucks or the white trucks. That was her way of differentiating between development projects. I explained to her I was doing (PhD) research, with neither. She asked me whether they (the trucks) would come back. I couldn’t answer, and therein lies the problem. “Country-led development” begins with communities being involved at every stage of a project. Ongoing community input during project design, implementation, and monitoring is needed for impact, local community ownership and sustainability. Developing “feedback loops” that facilitate two-way communication are key for building cultures of collaboration, adaptation and learning into international development programs. Valuing Voices sees data as another resource to deliver development, beyond serving the needs of donors and international non-profits.
The distance between our intentions and our reality
Too often, data is extracted from communities by development organizations, in order to evaluate how well they fulfilled the project, rather than communities evaluating how well international development projects supported community needs. The best projects co-design interventions and monitoring with communities. Too often communities have no mechanism to learn how their feedback during implementation or during evaluations led to real changes. Such feedback leads to community buy-in, as there is proof that their voices matter and that they are co-driving development. Outside of submitting a format report in a PDF to a donor, the development field has consistently inadequately retained the data from projects once they close, much less leave it in-country, and rarely return afterwards for follow-up evaluations. This further reinforces the idea that these communities are being exploited for resources or development experimentation, with no thought to long-term capacity, learning or sustainability.
Ever farther from this vision of collaborative partnership in project conceptualization, design, and monitoring is what happens once these short and long term projects are closed out. Due to budget restrictions and bureaucratic habits, too often the task of sustaining these projects is handed over to local partners, without funding, staff or data continuity. Cursed with “pilot-itis”, development initiatives too often lose sight of achieving scalability and sustainability. ‘Sustainability’ is often incorrectly defined only as whether the specific project got follow-on funding, rather than by asking communities what activities they were able to continue long after the project ends. Communities provide feedback only 1% of the time, missing great learning opportunities to be had from returning to assess these same projects 1-10 years later to see what was expected, unexpected and what could be sustained. We therefore never truly learn what had the greatest prospect for replicability and scaling elsewhere.
How do we really show that we Value Voices?
At Valuing Voices, we believe that community members are our true clients. We have identified two kinds of feedback loops that are needed to make international development far more effective.
- Country-Led Project Implementation Feedback Loops: Valuing Voices wants to create standardized methodologies and data collection processes that can be integrated into most international development projects to create feedback loops that continue working long after projects close. Valuing Voices believes key elements to developing collaborative feedback loops are:
- Identification of different feedback loop methodologies. Based on what is appropriate for the population, Valuing Voices identifies different methodologies to create these loops through a mix of traditional and digital tools. This means explicitly targeting the building, capturing and sharing of feedback during a specific project, to test and document different methodologies and create standard processes and infrastructure. This includes participatory methodologies and studies such as Empowerment Evaluation and “Who Counts? The Power of Participatory Statistics, as well as ways of evaluating qualitative Storytelling. We must of course protect our respondents as well.
- Use of a franchise approach to replicate and scale. The identified methodologies can be taught to local and in-country evaluators and development experts who can then “sell” those services to governments, local NGOs, international donors, and the private sector. Valuing Voices is a catalyst for this country-based franchise approach which strengthens national evaluation, capture and learning. It also allows for feedback loops to exist within a program as local evaluators provide feedback on impact and lessons for improvement. This leads to local empowerment, sustainability and aid transparency.
- Use of digital tools- capture once, share forever. Following evaluations on the local level, Valuing Voices then rolls up national evaluations by sector, analyzes them for what’s most sustained, and shares that learning around the world, influencing project design, implementation, funding and empowerment. In addition to generating feedback loops, this valuable feedback is entered into a curated database of findings for comparison across projects such as agriculture, livelihoods, credit, natural resources, health and education. By using structured data analysis, we can compare feedback data and actual behavior. We will look at whether we are capturing the right associated metadata (date, location, project) to contextualize feedback and bring forth lessons our partners can learn from, analyzing similarities and learning from them to improve program quality, organizational learning (of all partners) and sustainability prospects.
- Post-Project evaluation Feedback Loops: The eagerly awaited Local Systems: A Framework for Supporting Sustainable Development was published by USAID in May:
“More ‘ex-post’ evaluations — which are meant to determine the impact of a project after it is completed, sometimes years later… are needed to design and implement projects.”
This is great news, as in the last 24 years, USAID has not published a single post-project evaluation, and the World Bank has only published one. This could indicate that none were written, or none were seen as ‘publishable’.
Post-project (ex-post) evaluations should look at:
- The resilience of expected impacts of the project two, five, and ten years after close-out;
- The communities’ and NGOs’ ability to self-sustain those activities;
- Positive and negative unintended impacts of the project, both immediately in in the long term;
- Which activities the community and NGOs felt were successes which could not be maintained without further funding;
- Lessons for other projects on what interventions were most resilient which communities valued enough to continue themselves or for which NGOs valued enough to obtain additional funding, as well as what was not resilient.
These evaluations are rarer than snow in Sri Lanka. Feedback loops post-implementation are key to understanding the sustainability of projects and to improve transparency, efficacy and learning. Valuing Voices has a handful of examples of ex-post (post-project) evaluation done by organizations such as Mercy Corps and Plan International as well as bilateral JICA, but shockingly 99.9% of all development projects just don’t go back and check.
Voices to be valued
These are just a few examples of areas where we can value the voices of our communities, to be more effective and impactful with our efforts, and show our respect and value of communities’ input.
The time is ripe for growth, for these voices to be heard more loudly, telling us what they want development to be. Join us in advocating for funding for such feedback, join us as partners in our field sites, as partners in ICT database creation, as voices for communities.
Communities want their voices heard. Sustained development depends on it.
About the Authors:
Jindra Cekan, PhD is the founder of Valuing Voices at Cekan Consulting LLC. She has worked in international development for 25 years in participatory design, monitoring & evaluation and knowledge management in over 20 countries. Her PhD was in Mali, “Listening to One’s Clients” and she consults to non-profit, public (USAID), foundation and private sectors.
Siobhan Green, MA is the founder of Sonjara, Inc, and a member of ValuingVoices. She has worked in international development since 1992, and specializes in ICTs for development, knowledge management, and technology for monitoring and evaluation. Her master’s thesis was on “The Internet in Africa: Policy Perspectives and Approaches” in 1997, and she works with USAID and other USG clients, non-profits, and for-profit partners.
