How Do We Balance Measuring Outcomes with Measuring to Learn and Improve? (#SM_RE) | Beth’s Blog

How Do We Balance Measuring Outcomes with Measuring to Learn and Improve? (#SM_RE)

Measurement

On April 25, 2013, the Robert Wood Johnson Foundation (RWJF) will host a national convening of leaders from 11 foundations, as well as experts in evaluation, social media, and communications to develop measures for evaluating the impact of social media on foundation outcomes.    There’s also a Twitter chat on April 18th at 3 pm EST  if you want to talk about social media measurement.

I have been invited to participate and help frame the day with a brief presentation about the state of nonprofit social media measurement based on the book I authored with KD Paine, Measuring the Networked Nonprofit.     I’m very excited for this learning experience because over the two years my work as Visiting Scholar at the David and Lucile Packard Foundation has focused on going deep on facilitating peer learning for grantees on measuring outcomes for their networks with social media while balancing and upholding the principles of being networked.

I think sometimes there is a disconnect between measurement and the actual practice.   In some organizations, measurement is this thing done at the end to justify or validate social media.  In other organizations, those that are lucky enough to have M&E staff, there is a disconnect between measurement practices for programs and communications.    It is important to think of measurement integrated into everything the organization does – I’ve called it being data-informed.   For that to happen one has to look at a maturity of practice and where to take small steps towards improvement.    I developed one for networked nonprofits and social media called “Crawl, Walk, Run, Fly

Click to See Larger Version

These are the practice indicators for measurement – for three different areas – analysis, tools, and sense-making.   At SXSW,  I did a workshop with Carie Lewis and David Neff who assessed their organization’s measurement practices along the Crawl, Walk, Run, Fly continuum.  What was interesting, these very advanced social media practitioners have good measurement discipline, but it isn’t yet organizational or department wide in some cases.

The next step in using this framework has been to established quantifiable measures – which allows organizations to see progress.   I’ve also used this as benchmark for my training and peer learning projects so one could quantify transformation.  The framework also works well as a benchmarking tool so a group of  similar organizations working together as a movement can learn from one another.    This makes easier to embrace a key principle of movement building “sharing and documenting practices” that Rachel Weidinger shared during her session at NTC last week.

I think if I were to sum up social media metrics for nonprofits, I would say that best examples of social media practice and measurement are when social media is used for engagement with a purpose.   We have to look at what the result overall is – it is not just views for the sake of views or comments and likes for the sake of comments and likes.   It is about how social media (and other digital tools I might add) help an organization move towards longer-term outcomes in their theory of change or articulated goals.

Click to See Larger Version

This chart comes from Chapter 5 of Measuring the Networked Nonprofit where we discuss how to measure outcomes from social media versus activities.   The point is that there is sometimes confusion between activities and results.   One of the best parts of writing the book was being able to have long discussion with Katie Paine, the Queen of Measurement.    While specific outcomes and KPIs will vary from organization to organization and must be customized in context,   the chart is a generic list of potential outcomes that a nonprofit might see from social media and some generic Key Performance Indicators (the one metric that matters to measure to success).

There is also the issue of social media measurement tools.  I think the focus on analytics software distracts many people away from thinking about outcomes.  No matter what metrics we’re collecting, eventually we need to put them into excel and analyze them to get insights.   So, I’m constantly telling folks that the best social media measurement tool is excel.

It is very important for us not to forget the importance of building relationships, transparency, and experimenting.  Sometimes a too rigid focus on measurement can lead us away from being innovative and taking small risks or little bets – and even more importantly learning from failure.  Take for example how UpWell applies the learn start up thinking to their social media campaigns, using measurement.  This form of measurement and social media metrics are just as important as measuring outcomes.

And, I must also emphasize how important learning from failure is.   You can read my first Harvard Business School Review Blog post is about how nonprofits  learn from failure.    And, I’m sponsoring a little bet of a contest to get nonprofits to share their #npfail stories (details here).

In advance of convening on Social Media Metrics hosted by RWJF, there will be a Twitter chat on Thursday, April 18, 3-4 p.m. EDT, with RWJF staff Debra Joy Perez(@djoyperez), assistant vice president, Research and Evaluation, and  Tina Kauh (@tinakauh), program officer, Research and Evaluation and moderated by Kami Huyse from Zoetica.   Join RWJF online Thursday, April 18, at 3 p.m. Eastern Daylight Time for the Twitter chat using the hashtag #SM_RE.    This chat is an opportunity to engage with other meeting participants and a wider network on Twitter around current practices, assumptions, and opportunities to measure the impact of social media on philanthropic outcomes.   Stimulate your thinking about social media measurement, share your ideas and valuable resources, and learn from the experiences and perspectives of others.

 

 

11 Responses

  1. Mary Louise Avery CFRE says:

    Very interesting

  2. Your Crawl, Walk, Run, Fly continuum truly defines an optimal balance between measurement and practices. As an undergrad student studying social media techniques, this is very helpful to those of us trying to understand how to use metrics for learning and improving. Thanks for including your Key Performance Indicators and for emphasizing the importance of engagement with a purpose.

  3. [...] On April 25, 2013, the Robert Wood Johnson Foundation (RWJF) will host a national convening of leaders from 11 foundations, as well as experts in evaluation, social media, and communications to develop measures for evaluating the impact of social…  [...]

  4. Beth says:

    Hi Ashley,

    Thanks so much. All the material is from my recent book, “Measuring the Networked Nonprofit” http://amzn.to/measure-networknp

  5. [...] cause.  While content is surely an important factor, Beth Kanter explains in her recent blog post How Do We Balance Measuring Outcomes with Measuring to Learn and Improve? that the best example of social media practice is when it is used for “engagement with a [...]

  6. Ann Emery says:

    Beth,
    Thanks for your ongoing work in this area. I’d love to follow along with the April 25th RWJF convening. Will there be a livestream, will it be recorded and available online later, is there a twitter hashtag, etc.? Thanks!
    Ann

  7. Beth says:

    Ann Emery:

    You can find all the details here: http://storify.com/kanter/sm-re-social-media-metrics-chat I’m doing a session in the morning which I think will be live streamed. If you go to the storify, you’ll find links to the resource page and all the materials. You will love it….

  8. Beth says:

    Ann, thanks so much for sharing all your expertise in excel, etc and delighted you will following the conversation on Twitter!

  9. Mike Clargo says:

    Thanks for this Beth. I particularly like the crawl walk run fly maturity model. Is your focus purely on Social Media Metrics? It is just that I think your model applies to all aspects of metrics in an organisation and has a lot to offer them. We have been working with a tool which enables every individual to measure all aspects of their value-add on everybody else, and to add this up into team metrics, and organisational performance metrics for all aspects of how the organisation works together and serves its stakeholders. However, self-measurement appears to be something people espouse more in theory than in practice – but I think your model helps people think of simple steps, and perhaps that is the trick: start small and build. Thank you. (This link is how I get my own personal measurement feedback on my contributions: http://rr8.it/QC8 )

  10. Beth says:

    Thanks Matt, yes these are core organizational comptencies for measurment – measuring social or otherwise.

  11. Hafuswa Nakabiito says:

    thanks, these are hearts of organization ability for the balance measures and keep on

Leave a Reply