White Courtesy Telephone

Beige Philanthropy in the Crosshairs

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

Think of them as two armies battling against overwhelming indifference—and frequently against one another—for the soul of philanthropy.

One camp is arrayed under such banners as “metrics” and “evaluation,” and has a distinctly business school cast.  The other champions a style of philanthropy concerned with social justice, and seeks to expose the root causes of our social ills.

These are caricatures, but they’re not entirely without foundation, so indulge me for a moment longer.

We fail to appreciate how closely united these two camps are in their rejection of philanthropy as usual—“beige philanthropy,” as I like to call it.  Most striking, perhaps, is their primary point of contact: a shared concern for effectiveness.

Here’s how it works: Those in the Metrics camp argue that few foundations make any real attempt to hold themselves accountable.  Without measuring the effectiveness of their grantmaking, they’re unable to track progress against stated goals (if they exist).  Even if these grantmakers are able to see with their own eyes the good they help produce, it’s not clear that this good significantly outweighs the cost of producing it.  Accountable to nobody but themselves, they end up shortchanging the communities they aim to serve.

Those in the Social Justice camp come to exactly the same conclusion but via a different path.  They argue that most of the giving done by mainstream foundations is based on an incomplete or flawed analysis of what it takes to achieve goals like ending poverty or ensuring that all children thrive.  These foundations content themselves with triaging society’s victims, never wondering about the causes of their victimhood, and suspecting, perhaps, that they might themselves be implicated in the crime.

It’s an easy matter to strawman mainstream philanthropy, to claim that it’s grown soft on flattery and self-congratulation.  It’s a far better strategy, in my view, for those in the Metrics and Social Justice camps to make their arguments incontrovertible—to give their speculations the heft of widely shared conviction.  There’s also a lot they can learn from one another.

March Metrics Madness

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

I was recently invited by Bill Schambra, director of the Bradley Center for Philanthropy & Civic Renewal, to speak on a panel titled Metrics Mania.  Panelists were asked to comment on an essay commissioned from Gary Walker, the founding director of Public/Private Ventures, on the subject of evaluating social programs.  Here is the text of my remarks ...

Bill Schambra saw my role on this panel as the guy who when asked about metrics replies “metrics schmetrics,” but I’m going to have to disappoint him—to some extent.  Measurement and evaluation, when done properly, are not just a bit of value-added for philanthropic or nonprofit work, they’re absolutely essential.  Only a fool would disagree with that proposition. 

But here I mean not just the kinds of formal evaluations described by Gary Walker in his essay, but informal evaluation as well: the kinds of course corrections we naturally make when we embark on a project, take a false step, and adjust what we do accordingly.  Evaluation is not and should not be the sole province of the highly compensated consultant.  We evaluate all the time; our own eyes and ears notice things the most astute consultant will never notice; and we’ll often be our own worst critics.

Now here’s where the metrics schmetrics comes in, perhaps: We’ve written more nonsense about evaluation than just about any other subject in philanthropy.  Worries about evaluation, engendered in part by logic models the length of whale intestines, have become the math anxiety of the philanthropic world.

My general thesis—if I could call it that—is that from the perspective of somebody like Mr. Walker whose organization has been commissioned to conduct lucrative, large-scale evaluations of social programs (lucrative by nonprofit standards), the Impact Revolution might seem like a good thing.  But from the ground, from the perspective of many people working in community-based organizations, this so-called revolution has brought with it new sources of irritation, new ways of adding meaningless make-work to already overburdened nonprofit staff members.

It has not been a people’s revolution, in other words, but rather one championed by elites—like myself, I’m afraid— sometimes unable to see far enough beyond our own measuring sticks to understand the limitations of formal evaluation techniques, and the trade-offs in staff time and other resources that these formal techniques require.

The Axiology of Nonprofit Impact

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

In a celebrated 2001 essay, wordsmith Tony Proscio asked, “Are grantmakers a species of investor, building benevolent enterprises that produce a measurable return for society, or are they more passive enablers of good, seeking mainly to support those who pursue charitable ends by whatever path?”

He was commenting on the tendency of the foundation sector to import the language of business while remaining largely oblivious to its insights.  But his question still manages to provoke.  The debate over the “measurable return” of charitable enterprises rages on between the wielders of business metrics and the Luddites who oppose them.

The Nonprofit Roundtable of Greater Washington attempts to stake a middle ground with a new report titled, Beyond Charity: Recognizing Return on Investment.  The report includes the kinds of cost-saving arguments familiar to many of us: each dollar spent on Johnny’s education saves us tens of thousands of dollars in prison costs and helps turn Johnny into a productive citizen who pays taxes to support local schools and fire fighters.  But the report also attempts to highlight nonprofit contributions that are a little harder to quantify—strengthening community, improving the quality of life, stimulating reform.

I understand the need for return-on-investment arguments, but I also worry about their impact on audiences that might have a limited understanding of the civilizing effects of the nonprofit sector.

What’s the ROI, for example, on a provocative question that interrupts, if only for a moment, our relentless consumerism and reminds us of what we once aspired to become?  And where do we learn to distinguish between those cases in which metrics apply and those in which social goods are less susceptible to measurement? 

It used to be that notions like “character” and “good citizenship” figured prominently in arguments for maintaining the quality of a good public education.  Now it’s mostly about the dreary but important business of building a better workforce.

What do we lose when we attempt to convert every currency to a single coin?

Evaluation as Symbolic Capital

Evaluations too often function as ritual objects in our sector, tokens of a rite of passage: we did this thing, and it matters little what the evaluator heard or what he recorded in his report.  It matters only what we saw on our Vision Quest: our totem animal, a jaguar, running through the forest, bolting past knots of trembling stakeholders, drawing up to its prey: a program officer clucking his approval.

Some Hard Truths About Evaluation

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

Sean Stannard-Stockton, our blogger colleague at Tactical Philanthropy, poses these very difficult questions for his latest Giving Carnival:  How should social impact (or nonprofit effectiveness) be evaluated? How can we best understand the output of nonprofit or for-profit social enterprises? It is not enough to simply say that an organization is “doing good.”  How much good are they doing and how effective are they at turning “inputs” (donations and/or investment dollars) into social “outputs”?

I offer this modified version of an earlier post in reply ...

We seek a kind of scientific or moral certainty from a formal evaluation.  But it can provide neither.  The questions that funders most often bring to an evaluator—Was this program worth our $25,000 investment?  Should we continue funding it?—are questions only they can answer.  There’s simply no absolute scale against which an evaluator can measure the value of a philanthropic investment.

Here are some of the typical moves in the evaluation game:

WE:	Was this youth development program worth our $25,000 investment?
EVALUATOR:	Dunno. What are the kinds of outcomes you’d like from a youth development program?
WE:	Better grades in school.
EVALUATOR:	Hmm. Don’t see any of that. Would you settle for a lower dropout rate? I see some of that.
WE:	That sounds good. How much lower?
EVALUATOR:	I see a 15 percent lower dropout rate. Is that worth $25,000 to you?
WE:	Don’t know. Let me think about it.

The evaluator’s first job is to determine who is asking the evaluation question, and his second job is to discover what that person values.  Then and only then can he design an evaluation that attempts to detect those values.  An evaluator isn’t measuring the effects of a given program willy-nilly.  He can suggest various outcomes that we might look for from a given program, but, for the most part, he’s trying to detect the outcomes we think are worth seeing.

And so we’re at the stage where we’ve detected a 15 percent lower dropout rate for youth participating in our program, as in the example above.  Assuming it’s not an effect of how the young people were selected for the program, is the outcome—together with other secondary outcomes—worth what it cost to run the program?  We can try to compare this program with other programs we know, but they appear to be very different from one another.  Some have a mentoring component, others don’t; some include leadership training, others focus on academics.  The annual cost per student is $5,000 in one program, but $10,000 in another, but there are significant differences in the quality of program delivery.  At a certain point, the annual cost per student starts looking ridiculous—but what is that point? And is it the same for you as for me?

And so it goes.  What then is the value to society of a given nonprofit program or organization?  The simplest answer is that it’s whatever the market is willing to pay for it.

For those of us in the nonprofit sector, one of the hardest truths about evaluation is that the value to society, in dollar terms, of nonprofit work is as much determined by market forces as is the price of cheddar in Manhattan.  And we, the people who ask the evaluation questions—the funders, the donors, the board members—are that market.

Good evaluators know this.  They would gladly tell us, but often they intuit that we’d rather not know.  Frequently their role is to describe all the potentially valuable outcomes of a given program or organization and then retire from the scene, leaving us to ponder what kind of investment it might be worth.

The counsel most useful to us at that point might come from a priest or a philosopher, or, if we’re on a very tight budget, a Ouija board.