International Planned Parenthood Federation (IPPF) is the world’s largest sexual and reproductive health organization, leading a locally owned, globally connected civil society movement that provides and enables services and champions sexual and reproductive health and rights for all. Working with 152 partner organizations to deliver 175 million sexual and reproductive health services in over 45,000 service delivery points in 172 countries in 2015 alone means that systematically gathering client feedback is no small undertaking.
Like many health organizations, IPPF has traditionally relied on client satisfaction surveys to gather client feedback. Those client satisfaction surveys provide useful information but they can also result in an over-abundance of data that is difficult to analyze, interpret, and use to improve services. In addition, IPPF operates across a myriad of country situations, client populations and service delivery settings. To say that it’s challenging to identify a standardized client satisfaction survey that works across such an extreme range of contexts is an understatement!
In an effort to strengthen, streamline, and systematize feedback mechanisms for the purpose of continuously improving the client experience, IPPF has begun testing the Net Promoter Score (NPS) as an alternative approach in limited resource settings amongst clients with low literacy levels. The Net Promoter Score is a one-step customer satisfaction survey that more and more businesses are using to measure success; the NPS relies on a single question of how likely a consumer is to recommend a product or service on a 0 to 10 scale, usually followed with an open-ended question about the reason for the score. As discussed in a previous blog on “The Net Promoter Score for Nonprofit and Public Programs: Beneficiary Feedback,” social programs can learn from the business world by using tools such as the NPS as part of a feedback paradigm modified for the social sector.
To assess whether the NPS could be adapted for IPPF contexts, we conducted a pilot-test in two clinics in Mumbai, India, together with the IPPF Member Association Family Planning Association of India. We found that the NPS approach could indeed work amongst clients with low literacy levels, and that clinic staff were receptive to the approach. Clients were willing to be contacted to provide qualitative information about their experience, yet simply asking about the client’s reason for the score she gave did not yield much information about how service provision could be improved.
Finding that critical client feedback was difficult to obtain, IPPF brought the following question to a recent LabStorm:
How can we elicit meaningful, actionable feedback among low literacy populations in deferential cultures?
LabStorm participants came up with three overarching suggestions to help generate meaningful feedback that could be used to improve service provision:
- Define the feedback goal. The first step is to identify your client population and the specific questions you’ll ask. When constituents have a framework for providing information the feedback can be more robust.
For example, in gathering feedback, are you trying to identify why some clients are detractors? Or, are you more focused on moving clients who are passives to promoters? Does your clinic staff have ideas to improve services? Then maybe you’re looking to identify what clients like in the existing services so as to not inadvertently change them. Knowing what you want to know can help direct the feedback conversation so that your clients can respond to specific questions that respond to your feedback needs.
- Use NPS benchmarks to provide a framework for targeting feedback collection. In the pilot-test with FPAI, we sought answers to the Net Promoter Score in conjunction with collecting verbatim client feedback: first asking How likely are you to recommend this clinic to someone who needs similar services? and following that with What is the most important reason for your rating? However, we learned that it can be useful to gather the NPS rating data independently first. This creates space to explore the NPS data by client population and then use that information to identify specific feedback goals.
For example, in drilling down into the NPS data, you might see patterns emerge that help to direct your feedback collection efforts: perhaps you will see that clients aged 20 to 24 years old have a lower NPS than older clients, or perhaps you will see that family planning clients who choose an IUD have a higher NPS than those choosing an implant. Based on that information, you can identify the client populations and specific questions from which to gather in-depth qualitative feedback.
- Feedback should be a conversation, not a one-way street. To gather constituent feedback, think of the approach as a conversation, not a one-way dialog. Asking clients an open-ended ‘why?’ can leave them feeling uncertain and vulnerable, particularly in deferential cultures where a client may fear saying anything critical about a service that they value.
Knowing your feedback goal and using NPS benchmarking to identify where specific feedback is needed can facilitate a conversation with clients. By analyzing the data first, you can feed that information back into the conversation. For example, “We see that in this clinic, we have a lot of clients who think the services are ok, but they’re not enthusiastic about recommending the clinic. What do you think we could do to turn those clients into active promoters?” That takes the spotlight off the individual client and allows them to provide feedback in a less direct way that might feel more comfortable to someone unaccustomed to giving critical feedback.Approaching feedback gathering as a conversation also helps to address suggestions that clinic staff feel are not something they can do anything about. When asking open ended questions, clients might suggest adding a costly service for which resources are unavailable, or providing something that won’t be sustainable. With a conversational approach, staff could respond by saying, “We’ve heard that suggestion before and we’d like to be able to add that service, but we don’t currently have the resources to do that. Is there anything else you think we could do to improve services that might require less resources?” Or, “We’ve heard this suggestion from clients, but we’re worried that [this] might happen if we do that; what do you think we could do to avoid that happening?”
Another conversational approach is hypothesis-driven testing: where clinic staff brainstorm several ideas to improve services and asks the client for feedback on those different ideas. “We’re considering these three changes; which do you think would be better, or do you think we should do something else altogether?”
Successfully eliciting critical feedback rests on trying different approaches and creating a culture where staff feel comfortable gathering, and clients feel comfortable providing, feedback. The LabStorm message resonated with us and we are excited to address the inherent challenges associated with asking for feedback in deferential cultures by using a clear feedback framework as we continue to test the Net Promoter Score approach in different settings.
IPPF’s NPS work is made possible through the Support for International Planning Organisations Project (SIFPO2), implemented with the generous support of the American people through the United States Agency for International Development (USAID) under the terms of cooperative agreement no. AID-OAA-A-14-00038. The contents of this article are the sole responsibility of the International Planned Parenthood Federation (IPPF) and do not necessarily reflect the views of USAID or the United States Government.
Rebecca Koladycz is a consultant with a long history of working with International Planned Parenthood Federation. As the former Director of Organizational Learning and Evaluation of IPPF/Western Hemisphere Region, she led a team of evaluation, research and organizational learning experts who worked across departments and partners to strategically use data at the organizational, programmatic, and project levels. Throughout her career, she has partnered in the design, fundraising, management, and assessment of programs in healthcare and educational settings. At the core of her approach is a deep appreciation for the power of building collaborative relationships across all components of an organization to ensure continuous improvement and organizational development.
LabStorms are collaborative brainstorm sessions designed to help an organization wrestle with a challenge related to feedback loops, with the goal of providing actionable suggestions. LabStorms are facilitated by FBL members and friends who have a prototype, project idea, or ongoing experiment on which they would like feedback. Here, we provide report-outs from LabStorms. If you would like to participate in an upcoming LabStorm (either in person or by videoconference), please drop Sarah a note at [email protected].
