“The brain can only conceive one thing at a time,” Scott O’Neil, conductor of the Colorado Symphony, told his audience, as he explained how sophisticated musicians listen with their intellect, not just their emotion. He displayed a blank screen with two dots moving in different directions and asked the audience to focus on both dots together. Because of our cognitive limitations, he noted, “what you have to do in order to see both is you make them one thing. You make them a pair.”
Many discussions in philanthropy revolve around two approaches that, like O’Neil’s dots, seemingly move in different directions: giving from the mind vs. from the heart. Quantitative vs. qualitative data. Analysis vs. instincts.
Enter the flood of new and larger datasets referred to in various combinations as Big Data. By now, most of us have heard something about the promise and peril Big Data holds for the public good.
More generally, though, Big Data is simply the latest flashpoint in the tug-of-war mentioned above. And as many thoughtful people have recognized, the best approach usually is a mix: good philanthropy turns two (or more) decision-making inputs into a more nuanced pair.
Fewer people, though, have directly tackled the question of how to do so in ways that are practical for busy grantmakers and nonprofit staff. As philanthropies explore exciting new data sources, equal effort is needed to figure out ways to use any multifaceted set of data sources more efficiently – including the qualitative ones that will continue to play an important role.
The good news is that philanthropy is merely one of many sectors wrestling with this question. Several examples are in statistician Nate Silver’s 2012 book The Signal and the Noise. It may seem odd to cite one of the pop deans of quantification in this context, but Silver often focuses more on how data are aggregated than what the sources are. For example, he cites Cook Political Report election predictions, which combine qualitative (e.g., the candidate’s public speaking skill) and quantitative (e.g., polls and past elections) inputs into a forecast along a seven point scale. In particular, he lauds how the researcher who conducts candidate interviews “weigh[s] the information from his interviews without over-weighing it, which might actually make his forecasts worse. Whether information comes in a quantitative or qualitative flavor is not as important as how you use it.”
Similarly, in recounting baseball’s now-famous integration of statistical analysis described in the book Moneyball, Silver notes that “for a time, Moneyball was very threatening to people in the game” who thought that scouts might lose their jobs. “But the reckoning never came.” Instead, “teams are increasingly using every tool at their disposal,” using scouts to “find out information that other people can’t,” such as a player’s personality and “chemistry” with teammates. Many teams have even increased their scouting budgets, understanding that “the key is to develop tools and habits so that you are more often looking for ideas and information in the right places” and defining rigor by “the way the organization processes the information it collects.”
Philanthropy can do this, too.
The Cook Political Report and major league baseball teams may be particularly well-resourced operations. Any organization, though, can have “tools and habits” to improve its decision-making and learning.
For instance, we often work with clients to estimate “expected returns on investment” from opportunities among which they are trying to set priorities. Importantly, those estimates are not always quantitative – some are formed entirely from qualitative judgments – though we usually find that there is at least a bit of quantitative data that can inform decisions. Moreover, as described in a previous post on this blog, it is common to consider simultaneously many complex questions not directly captured by the initial estimates. Overall, expected return on investment is most useful as a tool to systematize information from disparate sources into a more digestible form for consideration – especially when integrated by habit into everyday decision-making discussions alongside other crucial inputs.
This is just one example of a more general point: as foundations such as Robert Wood Johnson have learned, employing simple and efficient means of facilitating comparisons and otherwise aggregating information is a powerful way to motivate positive change. In the debates mentioned above, as in so many others, it is easy simply to say, “All of the above!” What’s harder is acting on this mindset, especially for the many time and/or resource-constrained organizations in the social sector. Likewise, it is a truism that good philanthropy requires good gut instincts. But finding efficient ways to give ourselves a good, structured gut check – regardless of data availability and analytical capabilities – is an important step on the way to making every dollar count.
Going forward, we encourage the sector to devote more energy to developing practical ways for decision makers to consider numerous inputs efficiently, beyond simply debating the merits of various types of information or encouraging a mix in broad terms.