Philanthropy often seems to treat evaluation like most of us treat dentist appointments: something that we know is good for us, but rarely something to look forward to. Has it been six months? Make an appointment. Has it been three years? Commission an evaluation.
Being evaluators ourselves, it would seem self-serving for us to say that the field underinvests in evaluation. Truth be told, we at Redstone do think that is the reality – but we also believe that the evaluation community has underinvested in showing rigorously why evaluation is valuable. The onus is in large part on us consultants to show that evaluations (or strategic planning, or other “diversions” of grant resources) have high social returns, and under what circumstances those returns are greatest. It is insufficient simply to assert that an organization should spend X percent on evaluation or to settle for broad pronouncements of evaluation’s importance.
As a result, we are thrilled that the Hewlett Foundation has publicly shared portions of a board memo prepared by its two evaluation gurus, Fay Twersky and Amy Arbreton, that explores this topic in a bit more detail. As the report indicates, we were fortunate to work with Fay and Amy on pieces of the thinking summarized in the paper, and learned a great deal in the process (Fay’s paper on evaluation principles and practices remains one of the best resources we know of in this area).
Our sense, based on past experience and conversations with field leaders, is that there is no one right answer for how much to invest in evaluation. However, when evaluation is appropriate – such as for efforts that are large, uncertain, and risky, or where there is high potential for an evaluation to change practice across a given field – it can be one of the best investments a program makes. Yet while so many philanthropic initiatives meet one or more of those criteria, substantial investment in evaluation still is unusual. We tend to think, then, that evaluation remains undervalued.
Fay and Amy summarize the benefits of evaluation well: “We invest a little to learn a lot, and in learning we make our grant dollars more effective and more efficient…. The math is simple: if an evaluation whose cost is equal to five percent of the grants under review delivers results that are 10 percent better as a result of the evaluative information, it is a worthwhile expenditure; if it delivers a 50 percent improvement, the return is extraordinary.”
The work ahead of us is to illustrate that those returns are possible – even likely – with a high-quality evaluation.
Read Fay and Amy’s memo here.