White Courtesy Telephone

The View From the Heights of Arnstein’s Ladder: Resident Engagement by Community Foundations

The following article will appear in a special issue of the National Civic Review titled Philanthropy & Resident Engagement: The Promise for Democracy, available October 2013.  It will feature case studies, reports, and essays on the resident engagement experiences of public and private foundations.  This article was co-authored by Barry Knight, Director of CENTRIS-UK, and Albert Ruesga, President & CEO of the Greater New Orleans Foundation.

It goes by many names: citizen participation; community or resident engagement; “bottom-up” grantmaking; grassroots philanthropy.  For some community foundations in the US, it’s a pro-forma exercise; for others, a source of power and pride.  For a significant number it’s still little more than a fond dream, something wished for but somehow never attained.

When program officers and other community foundation leaders speak of stakeholder engagement, they often point to the desire to have their grantmaking interventions “informed” or “shaped” by the communities they serve.  These philanthropic professionals might go a step further—at least rhetorically—and seek the “meaningful participation” of underserved communities in the design and leadership of grantmaking programs.  The expression of these desires is sometimes motivated by a regard for democratic ideals or by a sincere respect for self-determination.  It might spring from a deep respect for fairness or an adherence to some form of the Golden Rule.  Stakeholder engagement might at times be sought to put a democratic patina on what are essentially elite decisions. 

Whatever their source, calls for resident engagement have become normative in the field, as reflected, for example, in the Grantmakers for Effective Organizations publication titled Do Nothing About Me Without Me: An Action Guide for Engaging Stakeholders (Bourns, 2010). The primary argument here is one about efficacy: Foundations that engage stakeholders are “part of a growing movement in philanthropy—a movement founded on the belief that grantmakers are more effective to the extent that they meaningfully engage their grantees and other key stakeholders” (Bourns, 2010; 1, emphasis mine).

And yet, when do our efforts to make our grantmaking less “top-down” move from the token to the meaningful?  Beyond the norms of professionalism, beyond the demands of our philanthropic technocracy, is there a moral dimension to the requirement that our grantmaking efforts be more citizen-informed and directed?

Fortunately for organized philanthropy, the issue of how to understand and assess resident engagement efforts is not new.  In 1969, for example, Sherry Arnstein published a powerful article addressing these key issues (Arnstein, 1969).  She proposed a typology, called the “ladder of citizen participation,” that has been widely used.  Her typology looked like this:

Ultimately for Arnstein, citizen participation is about citizen power.  The ladder reproduced in Figure 1 is designed to highlight the “critical difference between going through the empty ritual of participation and having the real power to affect the outcome of the process.”  The higher up the ladder you go, the greater the degree of citizen empowerment.  At rung four, for example, we invite citizen opinions but offer no assurance that these views will be heeded.  At rung five, we place a token number of the “worthy” poor on advisory committees and the like, but “[i]f they are not accountable to a constituency in the community and if the traditional power elite hold the majority of seats, the have-nots can be easily outvoted and outfoxed” (Arnstein, 1969; 9).  And so it goes as we climb or, in many cases, descend the ladder of engagement.  For in the world of community foundations, the kind of resident engagement that Ms. Arnstein holds up as the ideal—full citizen control—is rare or perhaps nonexistent, at least in the US context. 

In what follows, the authors attempt to answer three primary questions:

1. How do we understand the community foundation engagement of residents and other stakeholders and why is it important?
2. How well suited is the community foundation to the task of resident engagement?
3. What does successful resident engagement look like?

It’s certainly easy for community foundations to claim some form of citizen engagement.  But as we shall see, there are reasons why it’s difficult to do well.

Avoid Frankenthinking, Expert Suggests

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

In Mary Shelley’s beloved novel, Dr. Frankenstein was a visionary member of the scientific community of his day, who, sensing the terrible nature of his work, isolated himself from the world to conduct his experiments.  He was a god-like figure, a “modern Prometheus” breathing new life into inanimate flesh.

Imagine how a well-meaning member of the nonprofit or foundation world might have rewritten Shelley’s tale.  Perhaps, from this perspective, Dr. Frankenstein’s approach to creating human life was too top-down.  This is what ultimately led his townsmen to take up their torches and pitchforks to hunt down and kill his monster.

Suppose instead that Dr. Frankenstein had invited each villager to dig up his or her favorite body part in the churchyard and bring that to a community visioning process.  The creature, when finally stitched together, might have sported five arms and two heads and been unable to breathe for lack of a trachea, but it would have been their monster.

Unfortunately, research indicates that group brainstorming processes that involve simply eliciting and accepting the contributions of multiple participants, without examining these contributions critically, rarely outperform processes that incorporate a robust measure of respectful disagreement and debate. 

“Never, ever brainstorm,” cautioned Jonah Lehrer, the keynote speaker at a recent Grantmakers for Effective Organizations conference in Seattle.

And yet brainstorming, “consensus-building” processes are endemic to our field.  Every potentially bone-headed thing we’ve ever said is reverently recorded on a piece of flipchart paper now hanging in some facilitator’s office.  Being overly cautious not to offend our colleagues, we self-edit our “yes, buts” into “yes, ands,” as if by doing so we can bleach out the stain of our disagreement.

That is how we in the foundation world take what might be an extraordinary tale, a great advance in human knowledge, and turn it through groupthink into lifeless quiltwork.  These processes account for the myriad incoherent mission statements we encounter in our work, which are not so much mission statements as overwrought landscape paintings attempting to capture every inconsequential detail of some alien terrain.  What might have been simple statements of purpose are often yes-anded into meaningless word salad.

Inclusiveness is an important value in nonprofit work, I get that. Yet I often find that a regard for including all points of view in these processes does more to make us feel good than to improve our work.  We celebrate our consensus not fully understanding what the cost is to the communities we serve.

Philanthropy lacks the respectful, yet bracing, debate that is so much a part of the culture of the business world and of academia.  It would take a deliberate and sustained effort to shift our culture, but doing so might at least help us avoid compounding the sin of ineffectiveness with the sin of incoherence.

Grantmaking Strategy, Part 2: Of Mortals and True Strategists

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

Reality is that which, when you stop believing in it, doesn't go away.

— Philip K. Dick

Balm For The Soul ... If For Nothing Else

[Editor’s note: To read part 1 in this series of posts on grantmaking strategy, please click here.]

So somehow—using a combination of consultants, perhaps, and Ouija boards—we’ve answered the UR-Question and settled on a set of goals.  We’re now ready for strategic planning, part 2.

Before going on, I need to lay some cards on the table.  For a long time I’ve viewed most strategic planning work as a kind of hucksterism that has relied on the placebo effect far more than on any kind of intellectual rigor.  We attend many meetings, we convene many people, we contemplate a long series of pie charts, and at the end of this process we produce a document that garners some confidence for us and a nice check for our consultants.  Our newly minted strategic plan has an air of plausibility and so we plow ahead, putting aside our qualms about the hair-raising leaps in logic that purchased our assent.

Studies produced by very intelligent people chide grantmakers for a lack of strategy in their work, begging many questions about what constitutes a “strategy.”  In one such study, a strategy was defined as “[a] framework for decision making that is 1) focused on the external context in which the foundation works and 2) includes a hypothesized causal connection between use of foundation resources and goal achievement.”  The authors of this study went on to claim that a majority of the grantmaking frameworks they examined did not meet this definition.  This was to me an extraordinary claim, for I’ve never once in twenty-five years of nonprofit work met a grantmaker whose grantmaking “framework” was not strategic in the sense just described.  The definition sets a very low bar.  The authors of the study offered this by way of explanation:

What constituted a strategy under our definition? Here is an example that would meet our definition from a foundation that aims to improve children’s education. Its decision making is guided by a framework that is external to the foundation: “Provide grants to nonprofits that work with teachers to improve the quality of lessons.” In addition, the foundation staff could articulate a hypothesized causal connection between foundation resources and goal achievement: “Our grantmaking supporting improved teacher training stems from our analysis that under-qualified teachers are the biggest reason why children’s educational achievement isn’t improving fast enough.”

This stands in contrast to a decision-making framework for the same goal that would not meet our definition because it looks only inward at an internal process for providing grants: “Our grantmaking process starts with a letter of intent that is reviewed by a staff member and then a full application that is reviewed by two different staff committees.”

Are we to believe that none of the grantmakers in this second example, that nobody in this foundation’s leadership or on its staff, could, if properly engaged, give some account—any account—of how their grants might lead to improving children’s education?  This sounds to me more like a failure of communication than a real lack of strategy, at least as it’s defined by the authors.  It appears to me these folks simply didn’t understand the question.  Doesn’t the very fact that these grantmakers make grants to educators rather than to figure skaters suggest that they might not be the blithering idiots we take them to be?

Definitional issues loom large in the strategy debate.  If we narrow the definition of a strategy—perhaps by requiring that we articulate an elaborate theory of change to support our intervention—do we then heap scorn on grantmakers like Bill Somerville?  He argues, essentially, that funders should simply get out of their offices, identify who’s doing consistently good work in the community, and then give them some money.  Is Mr. Somerville’s m.o. unacceptably unstrategic?

Ironically, Bill Somerville’s approach—cavalier as it sounds—is in fact supported by a straightforward and, in my view, compelling theory of change.  It looks something like this:

1.  Each time I’ve given nonprofit leader X a grant and asked her to spend it wisely to make the world a better place, she has succeeded beyond my expectations.

2.  I will give nonprofit leader X a grant and ask her to spend it wisely to make the world a better place.

3. Therefore, nonprofit leader X will likely succeed beyond my expectations.

Nonprofit leader X inspires our trust because of her talent, her intelligence, and her ability to marshal resources and be a good leader.  These qualities abide with her and account for her consistently good performance.  It’s not irrational—nor, by extension, is it a bad strategy—to invest in people with qualities like these.

I would suggest that we leave the definition of what constitutes a grantmaking strategy wide open and simply assume that grantmakers have some idea of how their grants might lead to their desired effects.  We might need to expend some effort making these ideas explicit, but at the end of the day the question that matters most is not, “Do we have a strategy?” (most of us do) but rather, “Is it any good?” quickly followed by, “And how do we know?”*

There are ways, I believe, to address these questions without succumbing to the allure of theories of change and other forms of superstition.  I also believe that something has to be done to bridge the yawning gulf between our stated aspirations and our actual effects on the world. As grantmakers, we should either step up our game or get out of the way and let somebody else try.

 

Coming up: Grantmaking Strategy, Part 3: Taking a Harder Look At Some Common Social Change Strategies

_____

* I would also add, “Ultimately, is there any empirical evidence to support the claim that well articulated grantmaking strategies generally lead to better outcomes?”

Grantmaking Strategy, Part 1: The Ur-Question in Philanthropy

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

Without an eternal goal toward which time is necessarily tending …, history is a road leading nowhere.

— Charles Upton

I.  Many Things Pressing, One Thing Needful

Given a set of well-defined philanthropic goals—e.g., reduce world CO2 emissions by 10 percent over the next 20 years, provide quality health care for all residents of Levittown—we can, with a lot of hard work, devise plausible strategies for moving toward those goals.  It’s far more difficult for a donor or grantmaker to take a step back and answer the question: What in fact should my goals be, and why?

For those who control significant amounts of discretionary capital, this question is among the most vexing in all of philanthropy.  It’s the Ur-Question on which hinges so much that’s right and so much that’s wrong about contemporary philanthropic practice.

In spite of its importance, it’s also the question most often elided in the strategic planning work that grantmakers conduct.

How do independent foundations, which have almost complete freedom in the choice of what they can fund, rationalize their choice of grantmaking goals?

The space of potential goals can be narrowed to some degree by considering constraints on talent, money, and other resources, or by adhering to historical patterns of institutional investment.  But this narrowing of the space begs many questions and still leaves a mind-warping universe of possible goals for the grantmaker to consider.

Have you decided to end childhood hunger in your region?  This is clearly a worthy goal, and one with enormous practical and emotional appeal.  But is it tolerable for you to have well-nourished children with poor health outcomes, attending low-performing public schools, living in neighborhoods with significant gun violence?  You clearly can’t do everything, but why focus on this one thing rather than another that might yield greater benefits for society?  And greater benefits measured how?  Do you simply say, ‘We’ve got to start somewhere,’ and leave it at that?  Is that the best answer any of us can give?

I don’t believe it is.

A more satisfying answer takes us, I would argue, from the familiar landscape of logic models to the more unsettling terrain of philosophers and priests.  Addressing the Ur-Question in philanthropy rather than waving it aside requires that we pause to consider our cherished assumptions and, possibly, confront some potentially uncomfortable facts about ourselves.  This goes for grantmakers large and small.

II.  Uh, What Was the Question?

A very smart colleague of mine once devised an exercise to help individual donors get in touch with their inner-program officer.  (Relax: This is not as horrible as it sounds.)  I’m paraphrasing here, but she asked donors to consider something like the following scenario:

Mary is a single mother living with her two young children in Hardby, a run-down part of the city.  Unfortunately, Mary’s neighborhood hasn’t recovered from the ravages of recent hurricanes.  Homeless individuals and families live in some of the abandoned properties in the neighborhood which are also used by prostitutes and drug dealers. Escalating levels of drug-related violence in the neighborhood have made parents afraid to let their children play in the streets.

Mary’s daughter, eight-year-old Elizabeth, is a gifted musician but suffers from asthma and other health problems.  Mary fears that her son, 13-year-old Martin, will be “lost to the streets.”  He’s had difficulty at school.  Unlike his younger sister, Martin doesn’t like “book learning” but prefers instead to work with his hands.

Mary can’t work a regular 9 to 5 job because she can’t afford childcare for her two children.  To make ends meet, she takes care of her neighbor’s children during the day for a small fee and does piecemeal sewing work to supplement her small income.  She has often dreamed of a career in nursing, but simply hasn’t had the time or resources to pursue her dream.

Donors were then presented a list of some of the nonprofit organizations serving the Hardby community:

• Hardby Community Health Center: Provides free primary health care to Hardby residents
• Hardby Boys & Girls Club
• Microenterprises 4 All: Helps individuals launch and grow their small businesses
• Hardby Childcare Association: Provides childcare services to the indigent
• Childcare For All Coalition:  Advocates for higher quality and more accessible childcare services
• Young Promising Musicians Charter School
• Hardby Neighborhood Association: Organizes a neighborhood crime watch and organizes residents to advocate for better law enforcement and police protection
• Etc.

Here was the exercise: Each donor was given $1 million in Monopoly money and asked what kind of nonprofit investments she would make in this scenario.  The donor could choose to invest in one or several of the nonprofits on the list, or add her own.

As you can imagine, a very wide range of grantmaking strategies emerged from this exercise.   Some donors focused on the immediate needs of Mary and her family; others invested in longer-term solutions.  Some ignored Mary and focused on the children; others saw getting her into gainful employment as the key to all their futures.

I’ve conducted this exercise with many groups of donors, and what’s most surprising is that only once was I ever asked to specify the goal of the exercise—that is, to answer the Ur-Question. We can, with a little reflection, formulate the goal any number of ways, with each variation eliciting a different response. For example, we can ask the donor to invest the $1 million in such a way as to ...

… generate the most good for the most people

… address the root causes of the problems facing Mary, her children, and their community

… adhere to the Golden Rule

… maximize the social return on investment.

We can imagine the world we want to see.  For many in the West, it’s the Star Trek world of the 23rd century (minus the intergalactic warfare), in which everyone’s basic needs are met; everyone is engaged in meaningful employment; there’s significant time for leisure and contemplation; and bigotry has been eliminated.  For others it might be a world run by theocrats in which women are discouraged from pursuing careers, and single mothers are shunned. Unfortunately, our different visions of the ideal world are more often negotiated by violence than by reflection and debate.

Each of us can imagine the world we want to create.  What’s far more difficult is to formulate the goals and strategies that will propel us there.

III.  Doing To, Doing With

I’ve been reading Ralph Ellison’s Invisible Man, struck by the many parallels between the Brotherhood he alludes to in his book and the contemporary confraternity of progressive grantmakers.  He devotes several passages to three young boys—Zoot Suiters? Hipsters?—whom he describes as “tall and slender, walking stiffly with swinging shoulders in their well-pressed, too-hot-for-summer suits.”  They are “outside of historical time” because they don’t embrace the leftist ideology of the Brotherhood.  But they strike a nerve with the protagonist:

I got up and went behind them.  Women shoppers with bundles and impatient men in straw hats and seersucker suits stood along the platform as they passed.  And suddenly I found myself thinking, Do they come to bury the others or to be entombed, to give life or to receive it?  Do the others see them, think about them, even those standing close enough to speak?  And if they spoke back, would the impatient businessmen in conventional suits and tired housewives with their plunder, understand? … But who knew (and now I began to tremble so violently I had to lean against a refuse can)—who knew but that they were the saviors, the true leaders, the bearers of something precious?  The stewards of something uncomfortable, burdensome, which they hated because living outside the realm of history, there was no one to applaud their value and they themselves failed to understand it. … What if history was a gambler, instead of a force in a laboratory experiment, and the boys his ace in the hole?  What if history was not a reasonable citizen, but a madman full of paranoid guile and these boys his agents, his big surprise!

Among other things, I’m reminded by this extraordinary passage of the infelicity of words like “goals” and “strategies,” invoking, as they do, our many failed attempts to dam up and control the flow of human events.  I’m reminded of the distinction we sometimes make between “doing with” and “doing to,” vainly and arrogantly ignoring where the true sources of power lie. And I’m better able to appreciate Winston Churchill’s warning that “However beautiful the strategy, [we] should occasionally look at the results.”

Well, I’ve looked, and not only is Mount Zion still a hell of a ways off, but it appears to be receding into the distance.

 IV.  Good Actions in Bad Faith?

We would be less than honest with ourselves if we didn’t consider another possibility:  Just as local efforts to provide skills training to the unemployed can seem futile when the country as a whole has plunged into a deep recession, so can our collective efforts as grantmakers appear insignificant when contrasted with the vast forces affecting the communities we aim to serve.

For those of us attempting to fund significant social change, how do we rationalize our work to ourselves and others? Are we

• Convinced, in spite of the evidence to the contrary, that the revolution can be funded?
• Biding our time, ready to support the next great people’s movement when it comes along?
• Going through the motions, collecting our salaries month after month, too bought in or too blind to see that we’re not really making any headway?
• Resigned (as one of my colleagues put it) to taking a permanently adversarial position?

Or are we simply keeping hope alive because the alternative is too awful to contemplate?

 

Coming up: Grantmaking Strategy, Part 2: Of Mortals and True Strategists

 

 

Debunking Theories of Change, Part 2 of 2

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

This post is part 2 of 2 in a series devoted to questioning the utility of theories of change in nonprofit and philanthropic work.  To review part 1 of the series, click here.  You can download a full version of the argument by clicking here.

A Convention for Representing Theories of Change

Consider the program theory for a mentorship program:

(1)   A child’s close relationship with an adult mentor --> An increase in the child’s sense of self worth --> A reduction in the child’s susceptibility to violence

Here we read the arrow symbol (-->) as “causes” or “leads to.”  A theory of change, as outlined in (1), is essentially two linked explanations.  The first explains the increase in the child’s sense of self-worth on the basis of that child’s close relationship with an adult mentor; the second explains the reduction in the child’s susceptibility to violence on the basis of his increased sense of self-worth.  In this essay, I adopt, for various reasons I describe below, the conventions used by the great American philosopher of science Carl Hempel for representing explanations in the natural sciences.  Those conventions have us represent (1) as two linked syllogisms:

(2a)	1.	If I provide child A with a close relationship with an adult mentor, then child A's sense of self-worth will increase.
	2.	My program provides child A with a close relationship with an adult mentor.
	_______________________________
	3.	Therefore, child A's sense of self-worth will increase.

 

(2b)	1.	Child A's sense of self-worth will increase.
	2.	If child A's sense of self-worth increases, his susceptibility to violence will decrease.
	_______________________________
	3.	Therefore, child A's susceptibility to violence will decrease.

These two syllogisms  are linked in the sense that the conclusion of the first syllogism (“Child A’s sense of self-worth will increase”) becomes the initial premise of the second syllogism.

The syllogism outlined in (2a) is an example of what Hempel calls a “deductive-nomological explanation.”  Its first premise (“If I provide child A with …”) has the look and feel of a general law of human psychology: it tells us something about how normally constituted individuals tend to behave under certain ideal conditions.  Compare it, for example, with the following “lawlike” statement from physics: “If at time t = 0 we drop a ball from a height h above the earth, and we drop it in a perfect vacuum, and there is nothing to obstruct its path as it falls, then it will hit the ground at time t = the square root of (2h/g).”  The second premise in (2a) has a different character.  It’s an “observation” sentence that makes assertions about certain observable facts—viz., my program provides child A with a close relationship with an adult mentor; the mentor is well trained; etc.

Identifying each link in a theory of change with a deductive-nomological explanation as defined by Hempel has several advantages.  First, it provides a common language and framework for arguing about the utility of theories of change.  No longer need we suspect—as I often have—that people who disagree about theories of change are arguing past one another, that they understand “theories of change” in very different ways.  Second, Hempel’s formalism requires that we make explicit the laws and conditions that need to obtain in order for certain effects to follow from our social interventions.  The very schematic theory of change outlined in (1), for example, does not do this. By contrast, the theory of change described in (2a) is richer: it offers the evaluator a number of avenues for exploring the possible failure of a given social intervention.  If, for example, the mentored child’s sense of self-worth doesn’t increase, we can examine the reasons for this failure in two places: either adult mentorship (even under the best conditions) doesn’t tend to increase a child’s sense of self-worth—a question that can be explored empirically; or the program we’ve designed simply doesn’t instantiate the conditions necessary for the success of an adult-child mentorship relation.*

But enough theory.  Let’s look at some real cases …

 

Putting Theories of Change to the Test

Imagine that you are the executive director of an organization whose mission is to put a park within walking distance of every home in America.  You’ve done this work successfully in your city and now you’re approaching the XYZ Foundation for funding that will help expand your work to an adjacent town.  You send in your proposal confident that the XYZ Foundation will note your track record of success and fund your expansion.  You’re surprised when a program officer from the Foundation gives you a call and asks you to produce a theory of change for your program.  “You mean like our inputs, outputs, and expected outcomes—that kind of thing?” you ask.  “No, that’s a logic model,” he says.  “I want to see an articulation of the underlying beliefs and assumptions that guide your service delivery strategy and are believed to be critical for producing change and improvement.”   “Well, uh, our beliefs and assumptions are that our program will succeed in creating a park in Neighborhood X of Town Y and that this will induce the residents of Neighborhood X to use and enjoy it.  But we told you that in our proposal,” you answer him, somewhat testily.  “Do your best,” your program officer insists.

Although you suspect your program officer is deeply in the grip of an ideology, your need for funding drives you to comply.  You begin hesitantly:

(3)	1.	Every other time we've created a nice park within walking distance of residents' homes, they have used and enjoyed it.
	2.	Our effort in Neighborhood X will succeed in creating a nice park within walking distance of residents' homes.
	_______________________________
	3.	Therefore, residents of Neighborhood X will use and enjoy it.

You send this to your program officer and after several e-mail exchanges, you agree on this emendation of your original theory of change:

(4)	1.	If we create a park within walking distance of residents’ homes, and residents participate in the design of the park, and the city commits to its upkeep, then residents will use and enjoy it.
	2.	We will create a park within walking distance of the residents of Neighborhood X, and residents will participate in the design of the park, and we will get the city to commit to its upkeep.
	_______________________________
	3.	Therefore, the residents of Neighborhood X will use the park and enjoy it.

You receive funding from the XYZ Foundation and build the park in Neighborhood X, just as you described in your proposal, and just as outlined in your theory of change (which to you still appears completely superfluous—but never mind).  A year later, when you conduct an evaluation of your program you discover, to your great surprise, that the residents of Neighborhood X are not using the park.  After polling the residents, they tell you they feel unsafe in the park because drug dealers have started using it to conduct their business.  Soon after, you hear from your program officer.  “Your theory of change was wrong,” he e-mails you.  “Please refer to the attached document for the correct theory of change.”  You open it and find this:

(5)	1.	If we create a park within walking distance of residents’ homes, and residents participate in the design of the park, and the city commits to its upkeep, and residents feel safe in the park, then residents will use and enjoy it.
	2.	We will create a park within walking distance of the residents of Neighborhood X, and residents will participate in the design of the park, and the city will commit to its upkeep, and we’ll take steps to ensure that residents will feel safe in the park.
	_______________________________
	3.	Therefore, the residents of Neighborhood X will use the park and enjoy it.

So, OK, you now convince Town Y’s police department to station a police cruiser next to the park.  One year later, same problem: residents aren’t using the park.  It turns out that because of the rising levels of violence in Neighborhood X, residents don’t even feel safe walking out of their homes, much less to a nearby park.  You wonder if you’ll get another e-mail from your program officer with yet another version of your theory of change:

(6)	1.	If we create a park within walking distance of residents’ homes, and residents participate in the design of the park, and the city commits to its upkeep, and residents feel safe in the park, and residents feel safe walking from their homes to the park, then residents will use and enjoy it.
	2.	We will create a park within walking distance of the residents of Neighborhood X, etc.
	_______________________________
	3.	Therefore, the residents of Neighborhood X will use the park and enjoy it.

There are several things to be noted about this simple example:

1. The most relevant parts of your theory of change were already implicit in your original grant proposal.
2. Connected with this last point, if your program officer can’t “get” what your theory of change is simply by reading your proposal, he or she might simply be in the wrong business.
3. Even in a very simple example like this one, articulating the “underlying beliefs and assumptions that guide your service delivery strategy and are believed to be critical for producing change and improvement” can get ridiculously complicated.  In general, you cannot know ahead of time what the “relevant” beliefs and assumptions will be.
4. Your theory of change is not working for you: you are working for it.  It’s like a dead albatross around your neck that you occasionally remove and dip in formaldehyde to keep your funders happy.

Our problems are compounded when we look at more complicated examples.  In her article, “How Can Theory-Based Evaluation Make Greater Headway?”** Carol Weiss describes a program that gives tenants representation on the management board of their public housing project:

This program espouses multiple goals, such as improving the maintenance of the property, decreasing crime in and around the housing project, and encouraging residents to take greater responsibility for their lives in such areas as work and education.  These are ambitious goals and, to some, inflated hopes, but let us take the most realistic objective, improved maintenance of the project … (p. 503)

What’s interesting about the description that Ms. Weiss gives of this project is that many of us immediately share her intuition about which of  its goals is the most realistic.  Even without seeing a detailed proposal, or a theory of change, we can imagine ways that tenant representation on the management board of a public housing project might lead to improved maintenance of the property.  Ms. Weiss points out that there are multiple causal paths that can lead to this goal, and she sketches a program theory that I reproduce below as Figure 1:

 
Figure 1

What’s striking about this theory of change is its overwhelming complexity—and this in spite of the fact that it doesn’t begin to articulate all the underlying beliefs and assumptions that might be critical to the success of the project—e.g., How much power do residents wield on the management board compared with non-resident managers?  How constrained is the board by a lack of funding?  How much time can residents reasonably dedicate to the management of a complex property? etc.  Weiss argues that:

The small everyday steps in Figure 1 can be couched in social science language, such as sense of communal responsibility, access to authority, local knowledge, bureaucratic responsiveness, and so on.  In the conduct of the evaluation, the evaluator will need to operationalize these concepts and develop specific indicators that capture their essential meaning. (p. 506)

This sounds like an enormous undertaking for what starts out as a fairly simple project.  If, after a year, the condition of the property doesn’t improve, is it really practical to ask our evaluator to do as Weiss suggests?  How much would such an evaluation cost?  Would he or she test each of the branches in Figure 1?  And just as with our previous example, we can imagine a thousand reasons, not pictured in Figure 1, why the program might have failed.  How, in that case, would the theory of change we’ve so carefully crafted guide our evaluation efforts?

By the way, we can also imagine scenarios in which the property improves but for none of the reasons outlined in Figure 1!  One of the resident managers, for example, might have a brother-in-law whose contracting company does excellent work for half the price; or one of the resident-managers might be very successful at organizing resident work crews who dedicate one Saturday each month to repairs and upkeep; or the very presence of residents on the management board might inspire a local donor to endow and thereby double the public housing project’s maintenance budget; etc.  Real life is extremely complicated.  When human actors are involved, its twists and turns are almost always unpredictable.

It takes enormous effort to dream up a program theory like that pictured in Figure 1.  And as we’ve seen, it can easily turn out to be a completely fiction.  At this point we need to ask, whom does it serve and to what end?

Adding It All Up

To summarize:

1. An airtight, explicit theory of change, even for the simplest intervention, might be beyond the powers of any mortal known.  If you’re a funder, think twice before you require it.
2. What typically passes for a theory of change is a narrative of how certain causes can plausibly lead to certain effects.  If this is all you’re really after, then read the proposal.  Everything you want to know is likely already there.
3. Because a theory of change is more like a narrative than an explanation in the social sciences, it will be more compelling if it’s written by a good story teller on your staff.  This moves you from the domain of sound scientific practice to the realm of compelling literature. This raises for me serious questions about the weight we should assign to these narratives in grantmaking work.
4. Funders who do not require highly articulated, explicit theories of change from their grantees are not treating them as idiots savants.  They’re treating them as fully rational human beings who have had enough experience of the world to know which effects generally follow from which causes.

I am not here denying that theories of change, and the process of constructing them, have helped many people in the nonprofit world.  But while I myself have found it enormously useful to discuss my plans with my Jack Russell Terrier, I would never dream of requiring this of my grantees.  There are myriad ways that organizations can fruitfully plan their social interventions.  All will likely involve some anticipation of hoped-for effects, and most, I assume, will fall far short of fully developed theories of change.  Some interventions will be designed entirely on the basis of intuitions or gut feelings.  While this is not a style of program planning I practice or endorse, I’ve also not yet seen any evidence that its failure rate is higher than that of allegedly more “scientific” approaches to planning social interventions.

The acid test for me is this: Take a close look at your theory of change:  Does it tell you anything beyond what you’ve already included in your grant proposal?  Does it suggest to you ways that you might improve your work?  Given all the factors we’ve considered in this article, are you confident of its accuracy?  Is it absolutely required by your funders?  No?  Then, borrowing a suggestion from David Hume, commit it to the flames!  For it can do nothing but confound you and waste untold hours of your time.

-----

* Hempel’s formalism certainly has its limits, especially when one uses it to describe change processes that are not strictly linear.  And as Hempel himself acknowledged, special consideration must be given to so-called “probabilistic explanations.”  But these complexities only serve to underscore my point about the questionable utility of theories of change in nonprofit work.

** Weiss, Carol 1997. “How Can Theory-Based Evaluation Make Greater Headway?” Evaluation Review, Vol. 21 No. 4.



Debunking Theories of Change, Part 1 of 2

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

Editor’s note: A full version of the argument presented in this post is available by clicking here.

In the Spring 2010 issue of the Stanford Social Innovation Review, Paul Brest, President of the William and Flora Hewlett Foundation, took several of my colleagues* to the conceptual woodshed for their skepticism about “theories of change.”  As Mr. Brest and others understand them, theories of change include our beliefs about the causal paths that lead from our social interventions to our hoped-for effects.  These beliefs naturally shape funders’ approaches to social change. Grantmakers will not, for example, fund programs that attempt to reduce violence by requiring gang members to wear Styrofoam shoes, for this intervention would not be supported by any plausible theory of change.

If we agree with Mr. Brest that a theory of change includes one’s beliefs about the way the world will behave given this or that intervention, we might wonder what there is to be skeptical about.  Do the skeptics deny that grantees have beliefs about the effects of their social interventions?  At the very least, Mr. Brest is concerned that these skeptics feel it’s unimportant to require grantees to articulate their theories of change:

Schambra and Somerville argue that funders should focus their grantmaking on community-based organizations, and that funders should trust an organization’s leaders without requiring them to articulate theories of change ….  These skeptics are implicitly analogizing grantees to idiots savants—individuals who are able to do complex calculations … in their heads without knowing, let alone being able to explain, how they do it.  [p. 48, emphasis mine]

It should be noted that one of the interesting characteristics of idiots savants is that the complex calculations they perform in their heads are often correct.  If the nonprofit world harbored the analogues of idiots savants—executive directors, for example, who could correctly intuit the interventions required for lifting whole communities out of poverty—I would strongly encourage them rather than force them to give an account of their methods.  Despite their inability to explain how they arrived at their program designs, I would prefer that we went about changing the world the way they did, rather than the way most of us didn’t.

Mr. Brest’s claim that the skeptics alluded to in his article are analogizing grantees to idiots savants also needs to be challenged.  An idiot savant cannot explain to you how he is able, in under a second, to calculate the square root of a 55-digit number.  A talented nonprofit executive director, by contrast, can draw on a lifetime of experience and whatever research is available to him to give an account of the design of his social intervention.

In many cases, of course, nonprofit executive directors are working from implicit theories of change.  Since these implicit theories of change are ubiquitous, there’s a clear—but vacuous—sense in which theories of change are indeed as “powerful” as Mr. Brest claims.**  But this is not what’s at issue between him and the skeptics he alludes to in his article.  In writing about the evaluation of social interventions, for example, Mr. Brest claims that “Identifying the proper variables and metrics requires that one articulate the program’s goals and its underlying theory of change” (p. 48).  Even more forcefully than this, “Improving the lives of disadvantaged populations ... requires proven theories of change” (p. 47, emphases mine).

Should we really be requiring grantees to articulate their theories of change?  The simple answer, as I argue below, is that there’s little need for this requirement since most grantees already do a sufficiently good job of describing “the empirical basis underlying their intervention,” as Mr. Brest puts it.  They do this when they write their grant proposals.  The more surprising answer, perhaps, is that requiring grantees to produce explicit theories of change—beyond what they usually include in their grant proposals—does little to improve the art or science of grantmaking.

Highly elaborated theories of change are generally urged upon grantees by well-meaning people who have a limited understanding of how they function in the social sciences.  Because theories of change are often shrouded in the impenetrable verbiage of philanthropy, it’s also not surprising that most of us have little inkling of their theoretical and practical limits.

Speaking of practical limits, do you, dear reader, know with unerring certainty what a theory of change is?  If you do not, you are entirely forgiven and in very good company.  Take a little time to review the literature on theories of change and you’ll discover, as I did, that theories of change are everything and they are nothing.  Unfortunately, as I argue below, the “power” that Mr. Brest ascribes to them is largely the power to waste enormous amounts of your time.

The Radical Polysemy of “Theories of Change”

The term “theory of change” is used widely in philanthropy and other fields to describe a family of concepts whose members, upon close inspection, vary significantly in meaning.  “Theory of change” is a term of obscure origin, as far I can tell, associated in the minds of many with the culture and language of formal evaluation.  A grantmaker or grantee scanning the available literature on theories of change can hardly help but be confused by the exercise.  The GrantCraft publication titled Mapping Change, for example, opens by telling the reader not what a theory of change is, but rather what it does:

A theory of change describes a process of planned social change, from the assumptions that guide its design to the long-term goals it seeks to achieve. 

From this the reader might conclude that a theory of change is the narrative of some social change process with goals appended.  But would any such narrative qualify?  After all, “describing a process of planned social change” can mean writing a rambling manifesto, penning the succession plan for a drug cartel, or developing marketing guidelines for a smokeless ashtray.   The authors acknowledge this ambiguity when, several paragraphs into their article, they add:

What does “theory of change” really mean, in practice?  Grant makers who use the term may be describing anything from a detailed map to a general storyline.  What they agree on is that a theory of change is valuable if it helps them and their grantees understand the relationship between the problems they’re addressing and the strategies they’re using to get the work done.

At this stage, the careful reader might have some inkling of what a theory of change does and why it might be useful.  He might still have little idea what it is.  If the essential mark of a theory of change is that it helps us better understand how our efforts do—or do not—lead to our goals, then it’s possible that a glass of Merlot can qualify as a theory of change, for as Heraclitus reminds us, “It is better to hide ignorance, but it is hard to do this when we relax over wine.”

On one popular website the authors tell us that a theory of change “defines all the building blocks required to bring about a given long-term goal,” while on another we’re informed that it’s “the product of a series of critical-thinking exercises that provides a comprehensive picture of the early—and intermediate—term changes in a given community that are needed to reach a long-term goal articulated by the community.” In some publications they’re equated with so-called “logic models,” in others they’re not. 

One of the claims frequently made for theories of change is that they’re essential for helping us evaluate the effects of our social interventions.  As Mr. Brest himself informs us, “most evaluations of social interventions require understanding the theories that mediate between inputs, activities, and outcomes.” (p. 48)  Interestingly, many of the formal evaluation textbooks I researched didn’t use the term “theory of change” at all.  The closest cognates I found were the terms “program impact theory,” used, for example, in the 7th edition of Evaluation: A Systematic Approach by Peter H. Rossi et al., a popular textbook on evaluation; and the more common “program theory,” used in a significant corpus of books and papers by such authors as Carol Weiss and others.  Generally, these scholars understand a program theory to be a sequence of causes and effects, in which our interventions function as the instigating causes and certain hoped-for social benefits function as their ultimate effects.

To take a very simple-minded example, suppose we diagram the program theory for a mentorship program as follows:

(1)   A child’s close relationship with an adult mentor --> An increase in the child’s sense of self worth --> A reduction in the child’s susceptibility to violence

Here we might read the arrow symbol (-->) as “causes” or “leads to.”  But scan the theories of change produced by professional and amateur evaluators and you’ll find a significant range of choices regarding which types of causal connections to include (those drawn from individual belief-desire psychology, from the psychology of social movements, from rational actor theory, etc.); the necessary conditions one should make explicit (an inclination on the part of subjects to cooperate, a robust economy, favorable weather; etc.); and what unit of analysis to focus on (the individual, the family, the neighborhood)—among many other variations.

Given the general lack of consensus about what a theory of change is or what it should include, it’s not surprising that the skeptics alluded to in Mr. Brest’s article might fail to agree over its utility.  If we are ever to make any progress in our debates about theories of change, we will need a common language for describing and evaluating them. 

Coming up next: A definition, some case studies, and some damning conclusions

-----

* Bill Schambra, Bill Somerville, Sean Stannard-Stockton, and Jeff Berndt.

** Mr. Brest's article is titled "The Power of Theories of Chnage."

Hegemony By Proxy

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

A recent publication by Grantmakers for Effective Organizations, titled Do Nothing About Me Without Me, raises some vexing questions about the proper division of labor between foundations and the communities they aim to serve.

It’s not unusual for a foundation initiative to be “resident-led” up to the moment when the funder—rather than the residents—decide that facilitators, trainers, or other professionals are required for some aspect of the job.  At this point a “grassroots” project begins to lose its “bottom-up” character.

Along these same lines, when funders go about the business of identifying “indigenous leadership” in a neighborhood, they’ll often search for people who share their values and behave accordingly.  On several occasions I’ve seen resident leaders develop some very strong ideas of their own and begin to lose favor with the grantmakers who “empowered” them in the first place.  We might call this “funder hegemony by proxy.”

We seldom discuss these dimensions of stakeholder engagement, but should.  As grantmakers or donors we can’t have it both ways.  We cannot, in my view, claim bragging rights for supporting a community’s right to self-determination while we continue to call the shots and stack the jury.  If good sense requires the latter, then we need to own up to it and press the case against those for whom resident judgments and values and experiences are the measure of all things.

Seat of the Pants Philanthropy: Riffing on Strategy

       P   O   S   T   E   D       B   Y       A   L   B   E   R  T

1.  The Center for Effective Philanthropy recently published Beyond the Rhetoric: Foundation Strategy.  It should have made front page news, but it barely made a splash, even inside the world of foundations.  Here’s its primary finding:

We learned that even though most [foundation staff] interviewed believe that having and using a strategy increases a foundation’s ability to create impact, many do not use strategy in their own work. We asked respondents to describe the frameworks they use to guide their decisions. While some decisionmaking frameworks met our basic definition of strategy, a majority did not.

If I may be permitted to do a little translating: The majority of foundations do their grantmaking completely by the seat of their pants.  There’s no framework; no well articulated strategy for achieving desired ends.  The fate of some of our most vulnerable communities rests in the hands of grantmakers who do their work by feeling their way in the dark, as many of us do.

This sounds like a bad thing.  Is it?

2.  Keep in mind that for many people inside the foundation world, charity represents a kind of weak, emotional response to a social problem, while philanthropy, because it is strategic, attempts to get to the root of a problem and solve it once and for all.  Strategy is also what keeps our grantmaking from becoming a random walk through many bogus ideas about the mechanics of social change.

We use the word “strategy” frequently in our work.  But what is it really?

3.  Here’s the definition of “strategy” used by the Center for Effective Philanthropy:

A framework for decision-making that is 1) focused on the external context in which the foundation works and 2) includes a hypothesized causal connection between use of foundation resources and goal achievement.

What’s most striking about this definition is that it’s absurdly weak: Which foundation’s grantmaking doesn’t focus on the external context in which it does its work?  Which foundation CEO, when challenged, cannot articulate a connection between what his foundation funds and what his foundation sets out to accomplish?

Consider this example:  The Levittown Community Foundation aims to end gang violence in Levittown’s low-income neighborhoods.  Its strategy is to fund gang summits to achieve this end.  Foundation board and staff members believe that bringing members of different gangs together can help these gangs resolve their differences.

This has all the earmarks of a strategy, according to the Center for Effective Philanthropy definition: a framework for decision-making focused on the external context (Levittown’s low-income neighborhoods), positing a causal connection between what the foundation funds (gang summits) and its goals (the ending of gang violence).

And yet this strategy has almost no chance of succeeding.  It doesn’t address the issues that bring members of youth gangs into conflict; it does nothing to prevent young people from being drawn into gangs in the first place; it doesn’t incorporate what others have learned from failed attempts to sponsor gang summits; etcetera.

The Center for Effective Philanthropy definition is so broad that almost any kind of nonsense can count as a strategy.  And therein lies its strength:  Despite the fact that the definition is broad, it’s still the case that a majority of foundations fail to use any kind of strategy at all—good or bad—in their grantmaking.

The conclusion, lurking just behind the study’s primary finding, is that the foundation field is in disarray, having little effect, at best, and squandering enormous sums of money, at worst.

Is this a fair assessment?

4.  It might not be.  We have yet to determine empirically the correlation between explicit, well articulated foundation strategies and the production of social goods.  Foundations armed with detailed strategies and metrics might not consistently outperform grantmakers who work opportunistically without a rigid plan, who constantly adjust their tactics to a landscape that presents new opportunities and challenges.  Napoleon, for example, never developed a theory of change before going into battle.  There was no grand plan that connected the fall of Paris, say, to his winning of France.

Keep in mind also that a strategy only makes sense when the path to your goal isn’t direct, when there’s some significant uncertainty about whether your efforts will achieve your ends.  You don’t need a strategy for driving to the grocery store, but you might need a strategy for getting your school-aged child out of bed in the morning.  Likewise, a grantmaker who sets out simply to “improve the lives of children in our region” doesn’t need anything as complicated as what the word  strategy would suggest.  Just about any kindly act toward a child would achieve his desired end.

I suspect that many of the foundations studied by the Center for Effective Philanthropy had goals that were equally open-ended.  This fact, if it is a fact, would tend to inflate the numbers of the “unstrategic,” making the study’s conclusions appear much more dramatic than they are.  Why so many grantmakers would have such fuzzy goals must remain the subject of another post ...

The Art of Giving Well

It happens with surprising frequency.  A foundation CEO retires and the trustees replace her with a Very Important Person who knows precious little about grantmaking.

“Why do they think that people, who last week were in business or in government service, do not need preparation as they seek to tackle some of the toughest social problems of our times?” asks Joel Orosz in a recent Chronicle of Philanthropy op-ed.*  According to Orosz,

[F]ar too many newcomers to philanthropy behave as if philanthropic history began the moment they stepped across the threshold of their foundation.  They are certain that past foundation efforts have been mediocre at best and have little of value to teach them.

That seems especially true of donors who have come directly from great success in the business world and who are determined to use exactly the same methods in the world of philanthropy.  Thus old mistakes are repeated, and worse, old successes ignored.

There’s a clear difference between good and bad grantmaking, Orosz argues, and it takes significant experience and/or training to master the art.

Where facts are few, experts are many, says the sage.  But while grantmaking is a relatively young discipline, we’ve accumulated enough facts of the matter to celebrate the work of many and condemn not a few.

_____

* “Good Grantmaking Requires Expertise” in the September 25, 2007 edition.