“Imagine a ladder with steps from 0 to 10, with 0 being the worst case scenario and 10 being the best; on which step of the ladder do you currently stand?”
This question sets the stage for our most recent LabStorm. At first glance, it’s a simple question engaging citizens on overall happiness, fulfillment, and success. But if each component is highly subjective, how then can one country’s ranking be compared against another?
The Gallup World Poll paints a detailed picture of the social and economic issues affecting people in every corner of the Earth. Having conducted interviews in over 160 countries, Gallup annually covers over 99% of the world’s population. They get to the heart of what are good and bad jobs in a country, assess societal instability, and analyze the role of citizen engagement. From there, they help clients around the world make informed decisions on how to improve the lives of citizens around the globe.
Gallup asks the same set of core questions in every country, creating a comprehensive, globally comparable system. Last week, Neli Esipova brought the World Poll to a LabStorm to continue refining the most essential elements of a successful qualitative interview process: asking the right questions and reaching the right people.
The World Poll collects data spanning a dozen topics, from government and politics to general well-being through, in the majority of cases, in-person interviews. And yet, despite the exhaustive set of questions and intentional household model, some answers just don’t make sense. Gallup is interested in exploring cost-effective ways of using citizen feedback to dive more deeply into results data sets that surprise them.
Let’s reflect back on the life evaluation “ladder” question, and the complications of qualitative research. During the LabStorm, Neli shared the answer of a 60 year old woman living in a Southeast Asian slum. Squatting over her cookstove, the woman matter-of-factly answered the interviewer: “ten”. Why, they wondered, did she give the highest possible ranking to her quality of life? To her, the answer was simple: “I have a husband who loves me, a son who respects me, and enough money to dye my hair and give to charity – what more could I want?”
This story highlights the necessity of re-contacting interviewees with follow-up questions. Having a deeper understanding of the respondent’s opinions will aid Gallup’s ability to transform their responses into policy recommendations. Our LabStorm attendees had several ideas to refine how that is done, and help Gallup close the loop with survey respondents.
- Establish automated follow-ups. Technology creates avenues to increase responses in a cheaper and less time consuming ways. Automated voice calling (like IVR) reaction recognition software, and transcription software provide opportunities for designing follow-ups that might be less resource-intensive than training Gallup staff or contractors to conduct them. These follow-ups could focus on a specific question that arose from initial interviews and provide further insight as it relates to policy outcomes.With IVR, it could be possible to automate a series of questions asking respondents to react to initial outcomes; for example: “80% of respondents think X is true, do you agree?” In developed countries, reaction recognition software, like the kind employed by UX designers during website testing, reduces the burden of training new interviewers for every project. Instead, this kind of software could be used to capture respondents’ reactions as they take a survey for later bulk analysis. In this way, the follow-up conversations could provide a more nuanced understanding of respondents’ reactions and translate into better decision-making. Finally, transcription softwares could be used to more quickly analyze qualitative data for sets of keywords and emotional markers, in order to more efficiently recognize trends.
- Engage journalists. Journalists are trained in the art of the interview, and may be well equipped to engage participants in the interpretation of the data itself. Instead of Gallup needing to train an army of qualitative researchers to perform follow-ups, they may be able to partner with journalists, who already have the skills needed to engage in these kinds of conversations, and who might like the chance to break new ground on emerging or curious polling data. Organizations like Internews believe storytelling is a powerful way to connect citizens to the news and information they need to make their voices heard; partnering with Gallup could be an interesting and rigorous way of learning about people’s views on a wide variety of topics.
- Establish an alternative to classic polling. Recent political outcomes – like Brexit and the US Presidential election – could not be accurately predicted by with traditional polling measures. Organizations like Sentimetrics instead analyze information from people who do not know they’re being polled: namely, Twitter usage. Social media activity has the potential to predict societal change before it happens. Social media analysis would limit Gallup to testing in countries with high penetration of the internet, but it is a practice that, if established now, could gather data effectively as the use of social media continues to spread.
We are excited to learn more from Gallup as they test out new strategies for engaging their constituents. Want to stay involved? Reach out to us at [email protected] if you’d like to learn more from Neli and her team at Gallup. Contribute your thoughts below and check back here, on the Feedback Labs Blog, for updates on how we help drive this momentum forward.
Neli Esipova is the Director of Research for Global Migration and Regional Director for Gallup’s World Poll for 29 Eastern European and former Soviet Union countries. She has led Gallup’s groundbreaking research on global migration patterns while also managing more than 200 studies on various topics in Europe and Asia. She recently expanded her area of research to include global women’s studies. Esipova frequently presents insights from her research to international audiences. She presented at numerous well-being, migration, positive psychology, and leadership conferences. Esipova is the coauthor of more than 100 published articles. She has had several articles related to CIS region published in the Harvard International Review. As a director of global migration research, Neli also has published in Migration Letters, IOM migration research series, and she led the team that wrote the main chapter of IOM 2013 annual World Migration Report and IOM-Gallup 2015 How the World Views Migration report.
LabStorms are collaborative brainstorm sessions designed to help an organization wrestle with a challenge related to feedback loops, with the goal of providing actionable suggestions. LabStorms are facilitated by FBL members and friends who have a prototype, project idea, or ongoing experiment on which they would like feedback. Here, we provide report-outs from LabStorms. If you would like to participate in an upcoming LabStorm (either in person or by videoconference), please drop Sarah a note at [email protected].
