Development organizations often talk about creating a “norm shift.” Influencing the norms that dictate “how things are done”- in business, philanthropy, or government contracting- seems like a nearly intractable challenge. The Open Contracting Partnership (OCP), a global silo-busting initiative connecting governments, business, and civil society to deliver better goods and services for citizens, prevent corruption, and create a better business environment through open contracting, joined Feedback Labs in a LabStorm last month to discuss some tools that organizations might put in place to begin capturing and using feedback. They challenged that to create a norm shift we must first understand the space – and the people within that space – in which we, and those whom we work with, exert influence. We have to understand who we spend the most time interacting with, and how far their respective circles of influence reach. With that kind of map, they say, we can begin to gain clarity on a path forward towards specific strategic goals.
We talk a lot about feedback loops and data, but there are ways to get feedback – and create that kind of map – without actively asking for it. Instead of telling the system something, we can extract much of what we need to know, based on our own goals and using existing data.
The OCP extrapolated on MIT’s Immersion project to create a graph history of where their employees spend collaborative energy. Partnering with Thad Kerosky, they used email metadata from all of their staff members to create a graphic representation of their email traffic. The algorithm detects naturally forming communities, as indicated by e-mail correspondence. In this way the OCP could visualize what communities are the most active. Essentially, they could see exactly who is working with who.
In this way, the OCP could see if the reality of their communications matched an ideal configuration, as determined by staff priorities, strategic goals, and other key performance indicators. Are they spending too much time talking to one strategic community versus another? Is someone missing? If so, why? It is important to note that for the exercise that the OCP ran, they stripped aggregated email metadata of individual identifiers; so rather than this being an individual performance management tool, it rather lent itself to a broader understanding of overall organizational performance and activity.
But…what does this have to do with feedback? At Feedback Labs, we often talk about feedback in the form of perceptual data– information that is coming directly from constituents that describes their thoughts, feelings, and perceptions about a product or service. In this instance, the OCP realized that their email metadata could actually provide them with meaningful feedback about the strength of their organization’s relationship with key partners. The degree to which a partner was willing to be engaged over email with the OCP was actually a form of feedback from that partner. It was a type of revealed preference on which the OCP already had the data.
But once you’ve mapped these revealed preferences, then what happens? How might you use this information to inform organizational behavior?
Marc Maxmeister developed another tool to analyze a different component of e-mail correspondence. Zendoscope, a heuristic relationship manager, analyzes the ratio of e-mail correspondences, indicating who you are – or aren’t – responsive to. Whether we mean to or not, some people get a quicker email response from us than others. This too is implicit feedback on the value we place on that relationship.
It reveals hierarchy within networks of people. You are more likely to fire a rapid email to your boss, who is less likely to reply as quickly. Teams of people on the same level have roughly equal time gaps in their communications, on average. You can write often or you can write occasionally, but the time gaps are still roughly equal.
Marc converted these patterns into a dashboard for each of OCP staff person to see who they prioritize. The dashboard also associated people with the relevant goal for that organization, based on the organization’s theory of change.
Similarly to OCP’s work, Zendoscope is mining our g-mail to map our network engagements, to understand better where we’re investing our collaborative energy. Marc recalls that during an analysis session with OCP of this data, it came up that one of the most successful email trains was titled ‘beer?’ This led to lots of discussion with key stakeholders, and is an example of learning what works to build relationships – going out for beer is more strategic than you think! By providing a visual representation of where e-mail correspondences are with equal frequency – and more importantly where they are not – we can begin to figure out which strategic relationships are going well.
The results were compiled into a dashboard for real time tracking without any need for data entry. Together, these tools helped the Open Contracting Partnership become more aware of the revealed preferences of the communities whom they engage with the most over email, and how they were responding to those communities. Through this metadata feedback, they gained a greater ability to manage their relationships, partnerships, and communities. With this information, the OCP will explore if their actions reflect the organization’s theory of change.
But the question remains, is this “feedback” enough to catalyze a norm shift?
Although this metadata “feedback” cannot determine the quality of interactions, by analyzing the quantity it can show what communities are already mobilized. By locating the key influencers, we can determine who and what is needed to mobilize additional communities. Revealed preference could help determine key groups to target for further perceptual feedback work.
We’re excited to see where this work goes, and we look forward to updates from the Open Contracting Partnership team!
You can read more about the OCP’s work here.
Post your feedback below on how you envision these tools best practices. If you’re interested in pursuing the graph history for your own inbox go to https://immersion.media.mit.edu and if you want to know the responsiveness of your e-mail relationships check out zendoscope.djotjog.com. Feedback Labs members (Network Partners and Associate Members) are invited to propose and lead LabStorms. For more information, please contact Sarah Hennessy at [email protected].
LabStorms are collaborative brainstorm sessions designed to help an organization wrestle with a challenge related to feedback loops, with the goal of providing actionable suggestions. LabStorms are facilitated by FBL members and friends who have a prototype, project idea, or ongoing experiment on which they would like feedback. Here, we provide report-outs from LabStorms. If you would like to participate in an upcoming LabStorm (either in person or by videoconference), please drop Sarah at note at [email protected].
