Social Media: Ask What’s the Impact, Not the Return! | Beth’s Blog

Social Media: Ask What’s the Impact, Not the Return!

Measurement

Engage Media's Theory of Change

I’ve just finished co-authoring a book about measuring the Networked Nonprofit with colleague, KD Paine.   One of the concepts that we wrestled with was ”Social Media ROI,”   I don’t think it necessarily works for nonprofits, especially when it applied as a litmus test to see if there are financial outcomes — cost reductions or increased revenue.   Nonprofits should focus on continuum of value that incorporates both tangible and intangible benefits to integrating social media in communications or program goals.   Of course, the ultimate result is social change or social impact.

Don’t get me wrong,  I believe that for social media to be effective it must be aligned with organization’s program and communications objectives.   For nonprofits, those are not solely money or efficiency for everything that could be measured.    By aligning social media to organizational objectives and using measurement,  you can answer the question:   of all the ways we could be investing our resources, is social media the best choice to see the social change results that we want to see?

ROI is as an acronym for a business term, Return on Investment.   It was created in the 1920s as a financial measure developed by DuPont and used by Alfred Sloan to make General Motors manageable.  It is a flow chart that calculates business performance taking into account not only whether the company had a profit, but whether that profit was good enough relative to the assets it took to generate it.  Over those 90 years, the chart has been polished, refined and so deeply embedded in business thinking that Wall Street views it as the only legitimate means of measuring business performance.

Should we be using an industrial measurement model in a digitally networked  age?  Should nonprofits use  a narrow ROI definition in their quest to improve program results?    Why not apply a “Theory of Change” method?

Why Use A Theory of Change

As David Hunter asks and answers this question in his essay on how nonprofits can use a theory of change  in Leap of Reason by Mario Morino.    How can nonprofits walk the line in measurement between unsupportable optimistic claims and mind-numbing “over-measurement.?”     He suggests that way organizations can do this is to have robust theories of change that serve as blueprints for achieving specific results in well-defined domains.

theory of change is a specific and measurable description of a social change initiative that forms the basis for strategic planning, on-going decision-making and evaluation.   It defines all the results required to bring about a given long-term goal.   A theory of change lays out the results and preconditions on a visual map that lays out the pathway for social change.

How It Works

In its simplest form, A Theory of Change consists of a series of  ”so then” statements that guide the design and management of an organization and its programs to achieve socially meaningful outcomes.  It requires stakeholders to articulate underlying assumptions which can be tested and measured, and it also charts an incremental path towards results.

In their paper,  ”You Can Get There From Here,” James Connell and Adema Klem define a theory of change is simply a description of the why and how a program or initiative works.     A Theory of Change should be:

  1. Plausible (stakeholders believe the logic of the model is correct: if we do these things, we will get the results we want and expect);
  2. Doable (human, political and economic resources are seen as sufficient to implement the action strategies in the theory);
  3. Testable (stakeholders believe there are credible ways to discover whether the results are as predicted);
  4. Meaningful (stakeholders see the outcomes as important and the magnitude of change in these outcomes being pursued as worth the effort).

Integrating Social Media Measurement With Your Theory of Change

It begins with a “To what End?” discussion on your team that leads to the articulation of SMART objectives – with a clear chain of  ”so that” statements.    The intent is to get past simply saying “we’re going to get 1000 fans on Facebook.”  That might one early step along the journey.

If you keep asking to “what end?” and then layout the continuum of results, you can see the road ahead.    You can’t prove cause and effect, but you begin to see how to move along the path.

One of the problems I see is that social media is often seen as this “separate” thing – relegated to the intern or part-timer who works in isolation – the efforts are not aligned and connection with larger organizational goals.  Perhaps using a Theory of Change model can help with alignment beyond tactics or financial results.

What you think?    Does your organization use a “Theory of Change?” that identifies precisely how and where social media fits in?  I’d love to hear about it in the comments.   If you want to receive a notification about when the book is published,  please sign up here.

 

25 Responses

  1. Andrew says:

    At Nonprofit Finance Fund, our theory of change is built around cultivating financially-savvy nonprofit administrators, and since we’re engaged in advocacy and knowledge-dissemination, the aspirations of our social media are (where it is noblest) journalistic: We aim to share with the people who need to know what it is they need to know to be better-informed contributors to the well-being of their communities.

    And if, like us, you happen to be sharing traditionally ‘dry’ topics like nonprofit finance, you have the added burden of being interesting to people who may not know what they don’t know they need to know.

    Social media allows us to find those people based on what they’re talking about and be responsive, engaging and accessible to them when the opportunities for sharing information arise. Though we’re not there quite yet, we think there’s an opportunity to use social media to identify the nonprofit finance themes that emerge in rapidly-evolving news stories.

  2. [...] Social Media: Ask What’s the Change, Not the Return! I've been mulling over the Social Media ROI concept and how it doesn't necessarily work for nonprofits, especially when it applied as a litmus test to see… Source: http://www.bethkanter.org [...]

  3. Beth says:

    @andrew: Do you have a diagram or visual or have you formally looked at how social media supports your theory of change and how you measure it?

  4. [...] Social Media: Ask What’s the Change, Not the Return! [...]

  5. Good blog overall, Beth. But I don’t think that the SMART framework is what you want…I think you’ll find a Developmental Evaluation approach more useful. See: http://bit.ly/odi57i

  6. [...] Social Media: Ask What’s the Change, Not the Return! Engage Media's Theory of Change I’ve been mulling over the “Social Media ROI” concept and how it doesn’t necessarily work for nonprofits, especially when it applied as a litmus test to see if there are financial outcomes — cost reductions or… Source: http://www.bethkanter.org [...]

  7. Beth -

    What an excellent post. Thank you for highlighting what I feel is a very important issue for not-for-profits and associations overall. Although a firm believer in solid, grounded, evidence based action I also believe that if we don’t pay attention to the intangibles that we miss half the story.

    Well done.

    Shelly

  8. Beth says:

    Hi Shelly: How do we walk the line between needing to show “hard” evidence – and those intangibles that might lead to tangible success?

  9. Beth says:

    Steve Waddell: Huge fan of Michael Quinn Patton’s work – but don’t you think there is a difference between “SMART” and “TOC” or are they one in the same. I’m off to order his book.

  10. Andrew says:

    @Beth None of our diagrams have quite made it past the whiteboard yet, and I’d say we’re ‘formally’ in the hypothesis-testing phase. I’ll share as soon as I can!

  11. I think the challenge with the SMART and similar frameworks is that they are focused on relatively short-term and definable goals. Good for incremental and reform change. But in transformational change you must be very open to what emerges and integrating it into your change strategy…it’s about learning about the end-state, rather than defining it so clearly in advance. This requires systems change strategies that take quite a different measurement approach. Nevertheless, SMART and such frameworks can be useful for short-term and relatively simple change challenges.

  12. I think the challenge with the SMART and similar frameworks is that they are focused on relatively short-term and definable goals. Good for incremental and reform change. But in transformational change you must be very open to what emerges and integrating it into your change strategy…it’s about learning about the end-state, rather than defining it so clearly in advance. This requires systems change strategies that take quite a different measurement approach. Nevertheless, SMART and such frameworks can be useful for short-term and relatively simple change challenges. And therefore have a place within bigger systems change initiatives.

  13. [...] Social Media: Ask What’s the Change, Not the Return! [...]

  14. Cathy Clark says:

    Nice piece, and I am a huge fan of Theory of change (TOC) thinking, and have taught it for over 10 years to MBAs interested in social entrepreneurship. What you’ve described is what I teach as the first level of TOC analysis; the second is when you develop “assumption” trees about the assumptions that connect your “if-then” statements. Which of those assumptions are true and how would you know? Who else might be working on that piece with whom you could partner? That’s where TOC thinking gets really interesting and leads to ways to test your hypotheses, some of which can be done really efficiently by social media.

    Also, a small correction – Leap of Reason was written by Mario Morino of VPP, not David Hunter.

  15. Beth Kanter says:

    Cathy: Yes, Mario is the author, but David authored an essay in the book on TOC. I will make that correction – thanks

  16. Dear Beth

    Thanks for inviting me to comment. Difficult to tackle such an important topic that has been addressed nicely in a blog post like yours above.

    I agree with you that ROI for NGOs or non-profits is not a good measure at all. Surely it requires some kind of change on our part to do better and ROI for social media is a rather fuzzy concept (i.e. we all define it different, so what use does it have?).

    Instead, I suggest that we change by focusing on key drivers or operating metrics instead of financial numbers. For instance, if you have a fund raising campaign, the question is first how social media can help spread the word for the campaign and second, how this could affect the number of donors you will have.

    Another example for how this can work very well is IFB-Loewenmut e.V. that is active on various social media platforms, such as Twitter (has German and English feed). Social media is used smartly to get: more volunteers (e.g., experts who use their vacation time to donate their expertise and work with disabled children), raise more funds, get interested parties including companies and politicians to stay engaged with their different initiatives.

    How the above can be supported and improved with the help of using social media is maybe what we should focus upon more (e.g., getting a logistic concern to donate and ship all our equipment for free from Europe to South Africa… nice). Unless we stay in touch with those that are interested in our mission (e.g., helping disabled children), they cannot support our cause however they wish. Here social media can be an important tool to reach our target audience wherever they prefer to engage with us (e.g., webpage, blog, Facebook versus Twitter and so forth).

    ROI does not make sense here, does it? But sometimes we are overeager to measure nonsense because if I ask you what is the ROI of your mobile/fixed line phone or air conditioning, you would question my sanity would you not? So why ask it for Social Media? Like the phone it is a tool we need to do our charity work. What you think?

    Beth, thanks for another inspiring post.

  17. Beth says:

    Hi Cathy: Thanks for the correction. David Hunter wrote an essay in Leap of Reason, but Mario was the author.

  18. Beth says:

    Urs, thanks so much for your perspective. I love that you can bring in stories beyond the American borders.

    Anyway, I agree with you about capacity/process/developmental goals, but impact is important – and so are dollars to get the work done.

    I think the ROI business concept is too limiting.

    The issue is how do you get an organization to value these other less tangible benefits?

  19. Thanks for your reply Beth.

    Beth, if your social media activities have an impact on your key drivers …. the benefits are tangible.

    Thanks.

  20. AJ says:

    The ‘Theory of Change’ tie in is certainly interesting. . . I’m in agreement that if the primary goal is just ‘get 1000 Twitter followers in 20 days’ to no particular end, the organization’s social media ROI is probably going to fall short. Social media is becoming this buzzword for non-profits, where everyone thinks they need to be measured by the same scale, when in fact there is only a portion of non-profits that will actually benefit from aggressive/expansive use of social media. I work with an organization that has recently changed their online fundraising tactics to exclude some social media they’d been putting a lot of staff time into- with excellent results. Another I work with, though, is a non-profit performance hall that has seen enormous growth in both their concert attendance and online fundraising by spending lots of staff time doing social media outreach.

  21. [...] Kanter, author of a blog and two books on social networking and non-profit institutions, agrees: I don’t think [ROI] necessarily works for nonprofits, especially when it applied as a litmus [...]

  22. [...] What’s the value, not return? – Beth Kanter’s take on social analytics You might also enjoy…Social Business IntelligenceReal time monitoring only matters if your strategies matchBuilding systems and strategies for Social Business intelligence This entry was posted in Links, Social Business and tagged analytics, apple, Leadership, metrics, social analytics by Josh Dormont. Bookmark the permalink. /* [...]

Leave a Reply