Renee Bellis, Feedback Labs | May 13th, 2022
At Feedback Labs, we love hearing stories of how nonprofits and foundations have implemented feedback and we want to share those stories with our community to inspire and give helpful tips and tricks. We interviewed Dr. Sharon Asonganyi, who, at the time, was Director of Research and Impact at Thread and recent Crash Course participant, about how the Thread team worked to put the feedback loop into practice.
Thread is a Baltimore-based nonprofit working to combat social isolation and weave a new social fabric for Baltimoreans. Thread intervenes in the 9th grade by enrolling students who academically rank in the lowest quartile of their class and face additional challenges outside of the classroom. Thread commits to every young person it enrolls for 10 years and matches them with an extended family of up to four volunteers who provide tailored support like rides to school, weekend dinners, or navigating transitions to adulthood. This customized support provided by Thread Families is complemented by community collaborators who provide everything from tutoring or community service and employment opportunities, to legal, healthcare, and housing services.
In her role as Director of Research and Impact, Sharon managed three bodies of work including leading diversity, equity, and inclusion (DEI) efforts across the organization, implementing evaluation processes to understand impacts, and conducting research that generates evidence for or challenges the Thread model.
Sharon took the Feedback Crash Course sponsored by the T. Rowe Price Foundation with her colleague Erin Preshoot and used the Crash Course as a way to develop, pilot, and implement a feedback-driven monitoring product. The Learning and Development group at Thread was using a customer satisfaction survey after training volunteers. Responses to the customer satisfaction survey were positive, but the group was not getting information to help them take action on making improvements and the survey response rate was low. Here is how Thread used the Feedback Loop to design, implement, and take action on a new post-training survey.
Survey Design & Collection
The Learning and Development team knew the training program could be improved, so Sharon and Erin saw a Net Promoter Score (NPS) based survey as a way to understand the impact of the training program now and whether participants had intentions to use the knowledge gained in the training later. It was important that the survey be simple enough to be completed quickly and broad enough to collect a wide range of participant ideas.
The new survey used just three questions:
- The Net Promoter Score question – On a scale of 0 to 10, how likely are you to recommend Thread to a friend or colleague?
- The second question depends on the response to the first. If the person answered with 9 or 10, they are asked what Thread could have done for the person to give an even higher score. This gets at how Thread can go above and beyond. If the person answered with 8 or lower, they are asked what Thread could have done to get a higher score. By segmenting promoters (9s and 10s) from neutral respondents and detractors (8 and below), Thread can pinpoint clear actions that someone who gave a 7 or 8 score would want to see to become a promoter.
- To gather any final thoughts, the last question asks if there is anything else the person wants to add.
Previously, the post-training survey had been sent out after each training via email, resulting in a low response rate. During the Crash Course, Sharon and Erin decided to make a small change to their collection approach. Trainers were instructed to use a few minutes at the end of the training for participants to fill out the Survey Monkey survey and the response rate went up to 80%. Participants were given both a link and a QR code to access the survey in the way that worked best for them.
Before asking participants to take the survey, trainers gave a brief overview about how feedback was important to Thread so participants could understand why they were being asked for feedback and how it would be used. If a participant gave feedback during a session, they were asked to include it on the survey so all feedback was compiled in one place.
Analysis & Learning
Thread created a dashboard using Trello, a software that enables teams to organize data and tasks into cards, lists, and boards, so everyone on the training team could see the feedback that was coming in from the survey. The Trello board was split up by training instance so new and existing volunteer and staff trainings had separate lists. Each week the Survey Monkey responses were put into the Trello board and the overall Net Promoter Score (NPS) was calculated for each training
At the end of the month, the team reviews the data and closes out the meeting by answering two questions: what are you curious about and where can we lean in? These questions help the team think about how to use the data analysis to create a practice that leads to improvements in the NPS. The global NPS has consistently been around 71 since this survey practice was put in place in December 2021.
Through their data analysis, the Learning and Development group found out that new and existing volunteers have different needs regarding what kind of information they want during training. New volunteers want to know about Thread’s history while existing volunteers want to get straight to the content of the training. This insight will be used when creating future training.
The rich data is also shared across Thread to help other teams take action on feedback. The data showed that some volunteers have a preference for a virtual community event over in-person and this information was shared with the Events team. The Events team has moved to a balance of virtual and in-person community events to be able to engage community members who want to participate but don’t have the flexibility to come in person.
Thread uses a volunteer newsletter and Instagram posts to follow up with the community, let volunteers know that their feedback was heard, and share next steps.
What’s Next?
The Thread team has found that the Dialogue phase of the feedback loop can be challenging. Training participants go out into the community after trainings and it can be hard to reconnect with them to dialogue about the feedback data collected. The Research and Impact team has begun to brainstorm ideas for what dialogues could look like in the future and have identified this as an area with long-term growth potential.
Thread’s feedback example demonstrates that feedback collection can be simple and take place as part of the existing touchpoints and that having systems and processes in place to facilitate analysis and staff discussion about feedback can enable rapid learning and adaptation! We hope you are inspired by Thread’s example to create systems and used technology to improve your organization’s listening practices.
