Archive for the ‘Measurement’ Category

Metrics for Building, Scaling, and Funding Social Movements

Investing in movements or networks for social change is a strategy that some funders are using.  But, how do you measure the results?

Marino Morino, who wrote “Leap of Reason: Managing to Outcomes in an Era of Scarcity” pointed me to this recent report, “Transactions, Transformations, Translations:  Metrics That Matter for Building, Scaling and Funding Social Movements” by Manual Paster, Jennifer Ito, and Rachel Rosner with the USC Program for Environmental and Regional Equity and funded by the Ford Foundation.   The report addresses metrics for success for investing in broad field social movements or networked approaches to social change.  The report is written for funders and those on the ground doing the work in the context of networks, although it doesn’t go deep into practice.

The report captures a conundrum in measuring social change movements or networked approaches.    Outcomes for “wicked problems” can be easily counted – policies passed, housing the homeless, educating children.  But there are less tangible results such as “we changed the frame” or “we shifted members’ consciousness” which for grassroots organizers on the ground view as the vibrancy of the network .   The report lays out some new metrics for movement building – that are paths to the more easily counted tangible results and where the unit of analysis is the movement or network, not an organization.

As the report points out,  movement organizers are grappling with big questions.  It is less about how to raise funds for their organizations (although that’s important) but focused on the big picture:   What is the long-term change that we want to see?  What is needed to achieve it?  What roles do different organizations play?    The report identifies metrics to measure progress around this these questions – it asks and answers – “What exactly are the right metrics for today?

“Amazing large numbers of members, staging marches, and winning campaigns – all these remain important measures of a successfully growing movement.  There are, however, other equally important aspects that are often missed in the numbers alone, including the fundamental changes that a leader, organization, or community experiences through their involvement in organizing and advocacy.”

The report suggests that one needs metrics that represent two sides:

Transactions: These are markers, both internal (number of members) and external  (voter turnout).    While the data is not always to collect, such measures tend to be easier to track because they are more tangible.    But they only tell part of the story and skip over the richness of experience and momentum that can be a prelude to social change.

Transformations: These are important, but often “invisible” work.   They should how people, organizations, and movements have been altered through the collective efforts.   They can also show how societal or political views have been shifted.   These metrics are more qualitative in nature which makes them more difficult to define, capture, and track.

The report argues for using a combination of metrics to tell the fuller story of a movement’s success.  It goes on to define both transaction and transformation metrics in different categories for movement or network building which serves as the meat of the report.

  • Community organizing
  • Civic engagement
  • Leadership development
  • Alliance building
  • Campaigns
  • Research and policy analysis
  • Communications and framing
  • Media
  • Organizational Development
  • Movement Building

One of the most useful parts of the report is a two-page spread that illustrates sample metrics for transformations and transactions for each of these categories or a “metrics tool kit.”   The metrics are not intended to be prescriptive, but the reports recommends that movements need to co-create their metrics so the metrics transcend the organization.   For this to happen, organizations need to the space to begin to work together to build the common language and frameworks for these metrics to hold up against different approaches and models.

There is a category for traditional and social media and a couple of paragraphs in the report.    That’s exactly what the book, “Measuring the Networked Nonprofit,” that I wrote with KD Paine and will publish later this year is about it.

The report included a section on recommendations, including building the metrics tool box and building movement capacity to use metrics.   One of the resources that is mentioned in the book is a Progressive Technology Project’s database technology set up to track this work.   Here’s what the report said about capacity building:

Of course, metrics tools only work if you have skilled craftspeople who can use them effectively.   The presence of such metrics mavens varies across the landscape of movement organizations.    Metrics and measurements need to exist at every level of organization, but it makes a different when someone is in charge and helps organizations stay on track.   While community organizers often find themselves pressed to take the time to assess in light of daily crises and immediate problems, movement builders have learned the power of reflection and refreshing.  Metrics can help, and building them into organizational culture can be facilitated by having someone with responsibilities to make it happen – and to steep others in the new practices.

We spent a chapter or two talking about exactly how to put this into practice because is this a very important point.

The point that the report makes and I agree is that measurement needs to value both transformations and transactions – and that requires new attitudes and approaches.

Anyone out there using metrics to measure movements?

Update:  Special thanks to Victoria Vrana who shared the report with Mario who shared it with me … a networked approach to sharing of networked metrics!

The Fives Stages of Measurement Acceptance

Infographics, if done well, can be a useful way to boil down key principles, ideas, or themes.   They’re gaining popularity everywhere, including the nonprofit sector.   More and more nonprofits are starting to use them and it isn’t difficult to find lots of examples and curate them.

KD Paine, my co-author of the Measuring the Networked Nonprofit (due out in later in 2012), pointed me to this gem of a measurement checklist in an infographic format.   I like the positive frame they put on measurement malaise and the hat tip to Kobler-Ross.

1.  Denial:  You can’t measure it

2.  Fear:  What if we find out our social media didn’t perform well

3.  Confusion:   I don’t know how to measure

4.  Promotion:  Check out these charts and graphs!

5.  Accountability:  Connecting measurement to decision-making and getting better results

Where do you sit along the spectrum?

Quick Reflections from Social Innovation Summit 2011

I’ve been heads down working on my next book about measurement and networked nonprofits with KD Paine and our editor, Bill Paarlberg, sprinting towards our deadline.   Writing a book is always a learning experience and I can’t express my gratitude for having such a wonderful co-author and fantastic editor.      The combination of  learning so much about topic, plus the writing process has been incredible.  When you have a co-author who is a measurement geek, you measure progress towards your deadline.  We have a ten step drafting process, with stage 10 being perfection.     We have a visual chart that shows us the stage of each draft – so when we reached 80% milestone, my reward was to attend the afternoon sessions of the Social Innovation Summit 2011.

 

Even with a big deadline looming, I’m so glad that I did because the discussions about being data-informed, fail fast and learn, and social impact measurement are ideas that we discuss in our book.     As Shannon Schuyler, Head of Corporate Responsibility, PwC, who was the emcee for the event said,  ”Listen to these smart conversations and take notes.”    And, I did, in the form of curating tweets on these sessions:

Driving Social Innovation

Reach and Impact:  Where to Aim and Where to Start

Impact Investing

Driving Community Solutions:  Impact and Scale

One of my favorite sessions was facilitated by Akhtar Badshah, Senior Director of Global Community Affairs at Microsoft and it was called “Social Entreprenuer Showcase” featuring three founders of newly formed social ventures:    Andrew Yang, Venture for America; Melissa Kushner, Nonprofit-Share; and Wilson To, Life Lens.       Their projects were amazing, but reminded of quote from Dan’l Lewin, from Microsoft, “The day before something is truly a breakthrough – it is a crazy idea.”   Listening to these social entrepreneurs reminded me that while we talk about failure, impact, learning, scaling, and measurement as important ingredients for success – personal passion is also important.    And, I heard that in the voice of Charles Best, DonorsChoose.Org and John Wood, Room To Read.

And, I might also add, if you’re looking for a great holiday gift for someone who cares about kids, check out DonorsChoose.Org gift card or if you want to encourage your own kids to give back, check out Room To Read’s Zak the Yak Gives Back Book.

Jonathan Greenblatt, Director, Office of Social Innovation, White House,  shared a lot of insights about his new job.    He made several important points, which is at the heart of the book that KD Paine and I are working on:

“We manage what we measure, it is important to be accountable & transparent to your donors.  Success is progress over time”
“There is a tension between innovation and impact. New and shiny may not be the most impactful says”
“Focusing on metrics is important, but we need to humanise those numbers”

Nancy Lublin, CEO, DoSomething.org moderated a session called “Reach and Impact:  Where to Start and Where to Aim” with Paul Carttar, Director of the Social Innovation Fund, Corporation for National Service and Jaquelline Fuller, Director of Charitable Giving and Advocacy, Google.   It resonated a lot with the themes in my book, and no surprise that DoSomething.org is one of the featured case studies.

Nancy Lublin is really funny!   Her opening lines were hilarious.     That some treat the funding of “innovative” proposals as porn,  ”I’ll know it when I see it.”   And that the two criteria for innovative nonprofits is that have an engineer in the c-suite and use creative commons licenses.

Favorite quotes:

  • “Any organization that receives funding, should use impact assessment and have to make their reports public. No shame in failing.”
  • “Funding data analysts is not overhead”
  • “Don’t pimp your organization out to a funder.  If they funding guidelines don’t fit, move to the next source.”
  • “Non-profits must be focused + passionate about solving the problem vs. growing own organizations.”
  • “Everyone wants to tell the winning story, very few want to share the failure”
  • “Themes for non-profits use data in a smart way; share what u know openly; collaborate; and-it’s about solving the prob, not you!”

All in all, I left feeling inspired.   For more on the topics of using data for learning and impact, see the newest issue of  NTEN journal.

 

 

 

What’s in your social media measurement tool box and why?

Rob Stuart, who passed away recently and who was instrumental in the creation of the nonprofit technology movement always had a favorite question, “What tools are in your tool box?”   It was always a terrific conversation starter with nonprofit techies because knowing  the right tool for the job is important.     As I’ve been working on “Measuring the Networked Nonprofit” with co-author KD Paine, we’ve come to the chapter on measurement tools.  I sent out a query nonprofit tech colleagues who are social media mavens and ask that age old question, “What’s in your social media measurement tool box?”

Because specific tools come and go and always are changing, we wanted to avoid writing something that would be out of date as soon the book came out.     The important thing is to pick the right tool for the job — and that job is collecting data to measure progress towards a goal or objective.

Why is that so hard?

Social media measurement tools give you data.  The problem is that data everywhere!   Almost every click we take, it leaves a data trail of data that can be captured.   Measurement tools or perhaps more accurately, social analytics tools collect that data.   The challenge becomes figuring what data really matters.

As much as we’d like it to be so,   the data we need to measure progress on goals will come from a variety of sources and tools based on our objectives and KPIs.  There isn’t really any one-stop shopping, although some of the tools that combine social media measurement with social management are attractive to nonprofits because of their simplicity. (One place to look).

Social media metrics mavens know that both qualitative and quantitative data is needed.  Quantitative is counting or the numbers — all those pretty charts and graphics.  The qualitative comes in the forms of comments and can often the why the numbers are moving in a particular direction

Measurement should not, ideally, be driven by a tool.   It should be driven by what you want to learn.     As soon as you have articulated a SMART objective and KPI, it should guide you to what data you need to collect and transform into insights   That prevents you from collecting data you don’t need and being overwhelmed.  If you create the discipline to collect data weekly, then it won’t be overwhelming.   Don’t wait to collect a year’s worth of data in a week.    Finally,  avoid getting into data collection and analysis ruts – and evaluate your approach.

Sometimes we think that because we’ve purchased a fancy software package, that it will give us actionable information with just one click.   That usually isn’t case.  Measurement and monitoring tools are great at scooping data from the social web, but the act of sense-making and getting aluable insights actually requires additional work.   That work isn’ t root canal – it’s actually fun.

KD Paine offers a great framework for matching tools with data collection and analysis tasks.

If you are measuring sentiment, then you’ll need a content analysis tool.     If you are measuring attitudes, perceptions, or behavior change, you’ll need to use survey research tools.   If you are measuring reach, engagement or action, you’ll need an analytics tool.

Nonprofit social media peeps identified four categories of tools as their favorites:

  • All Purpose:  Spreadsheets, both google spreadsheets and excel spreadsheets
  • Measurement and Management Combo Tools:  Sprout Social, Thrive, and Spreadfast
  • Analytics Tools:  Google Analytics,  Facebook Insights,  Twitter Tools (Socialbro, Hootsuite, Bit.ly, Twitalyzer)
  • Content Analysis Tools:  Radian 6 and Netvibes

Here’s a round up of specific tools and tips:

Spreadsheets

Social media metrics mavens are members of the Spreadsheet Association Society, whether it’s a google spreadsheet shared online or Microsoft excel spreadsheet.     Shonali Burke says, “Especially for nonprofits with limited resources. As long as you are tracking data points consistently, you can use simple correlation formulas to see how your efforts are impacting your outcomes.”   Burke also likes RowFeeder to collect Twitter and Facebook metrics because it is very easy to use and not expensive.

Holly Ross, NTEN, also loves spreadsheets.  “I pull in metrics from all over the web to get instant snapshots and create graphs that show changes over time.  While it takes a little bit of time, the tool is free.    Ross says being focused is very important because the more you track, the more time it takes.  She advices subjecting your data to the so what test.   When she isn’t playing with spreadsheets, she uses google analytics, tracking the traffic to their site from social sources, where they go and what they do.   [Google Analytics recently integrated social analytics tracking and multi-channel funnels]

MMM Tools: Measurement, Monitoring, and Management

A number of nonprofit social media mavens reported their favorite type of tool is a MMM or Measurement-Monitoring-Management combo tools.   While they are not free, they’re not beyond nonprofit budgets.  These tools combine what’s needed into one application.    They also say that a single MMM tool won’t solve all your needs and that Google Analytics is another must have.

David J. Neff,  Lights. Camera. Help., likes Sprout Social.   He says, “It measures all the small stuff and can give custom reports as well. It also at a glance let’s you see the 30k foot view.”    He also likes to use TweetReach.  “Both are easy to understand, export reports and good for mid level and senior managers to look at without heads exploding.”

Danielle Brigida, National Wildlife Federation, also includes Sprout Social among her favorite tools because it is helpful to track the performance of content across channels.   She turns to  Small Act Thrive for managing relationships and tracking keywords, for example “birding.”   She has discovered that there isn’t a Swiss Army Knife for social media measurement, monitoring, and management tools and will cobble together data from different tools into a spreadsheet.

Caryn Stein, Network for Good, also uses Thrive.  She says, “We track social reach, engagement and track keywords and mentions. Like many other tools, it’s a quick dashboard that can help guide your efforts and add a layer of context over general impact numbers provided by other analytics suites, such as Google Analytics.”    Her tip is to have an analytical framework before begin and be consistent.   If you are just getting started, pick one or two indicators.

Shari Ilsen, VolunteerMatch, prefers Spredfast which is another combo tool that tracks their activity (inputs) across channels as well as reach and engagement.   What she likes best is that the tool can help them tag specific campaigns and do more fine-tuned analysis.   Shari warns that there’s a lot of data points to collect,”so choose a few things to monitor or track, and then develop a workflow that will allow you to gain a big picture view.  You don’t need the bells and whistles if you’re just starting out.”

Kami Huyse, Zoetica, points to Lithium, a monitoring and measurement tool.   She likes it because it can track sentiment fairly well and the backend is easy to use.    She warns, “You can’t rely on a single tool. My second favorite is Google Analytics because that’s how you track actions.”  Huyse says it is important to set up a dashboard workspaces.

Analytics Tools

Social Media nonprofit mavens also like free analytics tools.  There are many to choose from that will collect metrics from Facebook, Twitter, and other channels.   Danielle Brigida says that the tools are often changing and don’t be afraid to explore, however, don’t let it get in the way of being a consistent measurer.

Debra Askanase, Community Organizer 2.0, feels strongly that smart use of Google Analytics as a first step will help nonprofits take that first step on the road to being a metrics maven.  “I help clients tie their social media actions to their web site.  We identify where we want social media referrals to go on the web site and track how many are getting to those pages.”   Askanase says that Google Analytics is a must-have for campaign tracking, especially the annotation feature.

Askanase’s other favorites include bitly.com and Metricly – all for tracking her network on Twitter.  Susan Tenby, TechSoup, loves Social Bro which helps her identify influencers and makes sure that her Twitter interactions are spreading across their network.   She checks three times a week.   Tenby, a maven of free and low cost tools, also likes Twitalyzer and Nutshell mail because they are really easy to use.

JD Lasica, Socialbrite.org, says his favorite is Google Analytics. “It’s the grandpappy of measurement, and it measures not just website referrals but social media kahunas you should be paying attention to, too!” Dan Portnoy,  Portnoy Media, loves Facebook Insights, but because it can track so many data points – he is very choosy about he collects and analyzes.   He also uses Hootsuite for tracking Twitter and Google Analytics.

John Haydon is a fan of Hootsuite.   He uses Hootsuite to build custom reports that include Facebook Pages, Twitter and Google Analytics.  Says John, “The reports are very simple to understand, and are automatically emailed to me.”

Monitoring Tools

Some social mavens also mentioned monitoring tools as their favorites which are useful for content analysis as KD Paine notes above.     Allen Gunn,  Executive Director, Aspiration likes NetVibes for monitoring.   Gunn says,  “I’m mainly looking for activity around tags and who to thank for mentions. And, the prices is right: free.”

While not free,  another favorite tool mentioned as Radian 6.   Amy Sample Ward, NTEN, uses to see how much they are engaging with the sector and whether NTEN is tiny fraction or sizable porition of the nonprofit technology conversation on social channels.         Amy advises proioritizing your data because Radian 6 can give you a lot.    For those getting started,  Amy recommends starting with free tools to understand the work flow.   Rachel Weidinger, concurs,  “Start a measurement habit, don’t wait for a budget for fancy tools – senior leaders rarely see it as important investment.    Start with building your practice first.”

What are your favorite social media measurement tools?  Why?

 

When Is One Million Fans on Facebook Worth More Than A Million Bucks?

HSUS Director of Emerging Media Carie Lewis Celebrates

Answer:  When they are engaged and ready to take action for you!

Earlier this month. the Humane Society of United States reached 1 million fans on its Facebook Page.   Says Carie Lewis, director of Emerging for the HSUS, “Although we prefer not to focus on numbers as a measurement of social media success, one million fans is a huge landmark that we are celebrated with our online community.”   We know that animal welfare nonprofit rule on Facebook, but how did HSUS do this?   Says Carie, “By listening to concerns of our fans, producing content that people want to share and making sure every post provides value to our fans and to the animals they care about.”

 

Facebook alone doesn’t do get an engaged crowd that takes action.  As  NTEN points out, the campaign include integrated tactics:

  • They have a custom landing tab on Facebook for the campaign, tied to a contest: they’re asking their fans to share their pets’ stories and photos. Everybody who uploads a photo gets a discount coupon to their online store.
  • Their CEO shot a video and blogged about the milestone.
  • They issued a press release and will be sending a message to their full house file.
  • And, there’s a banner on their home page.
  • They highlighted the campaign in their email newsletter – before, during, and after the event.
  • Click to see full infographic

    Like all rock star nonprofit social media mavens, Carie Lewis is a curator of social media metrics.   She and her team build their integrated strategy around results metrics.   Says Carie, “We look at three things: actions taken, donations made, and customer service wins. That’s also how our department has been able to obtain more resources to handle the volume we have.”      They also have metrics for specific campaigns and Carie is very good at tracking tactics against data to improve and get better results.

    For this campaign,  they wanted to create a celebration so that fans could engage and participate in the fun.   They wanted to create a personalized experience that makes the fans feel like they are a part of something really great that’s why they created a video and an opportunity for their fans to share their photos of their pets and why they love them.

    Some counting metrics they captured were:   # likes, # photo submissions, # mobile submissions, # tab views, # video views, # shares

    Says Carie, “We from our past experiences that we need to make it as easy and simple as possible for people to participate.  And good news works best, people love to celebrate and feel a part of something.”

    Further, Carie is a master of capturing data and reflecting on what works and what doesn’t – channel by channel.  Take for example her very useful tips about how to activate Facebook Fans to share and take action.

    What’s the secret to measuring integrated campaigns?   Carie says that  getting everyone on the same page is crucial.   “We have a daily noon meeting where someone from each end of communications – website, email, social, PR, and video – gives a 1 minute account of what they are pushing out that day. We also have a weekly cross-sectional meeting that talks about longer term projects. About 2 months before we launched the campaign, I presented my plan to the cross-sectional meeting, and got everyone on the same page. That was key.  There were things that I never would have thought about – photo disclaimer language, working around our CEO’s crazy schedule for taping the video, etc. And there were a lot of great ideas that were born that I never thought about, like creating an infographic about our Facebook fans. Make sure you know what resources you have at your disposal.”

    Conversely, when they are helping a campaign promote their work using social media, they have a “menu” of social media tactics that we review with them, letting them know the options and use cases for each.  As Carie notes, “This has really shown others that don’t work in social media everyday that a Facebook post on HSUS’ page isn’t always the best answer. Social media is no longer an afterthought in communications at HSUS.”   Here’s an example of some of that template, notice they ask for a screen capture of the action and to record any feedback.

    The biggest thing I’ve learned from measuring is to write down the metrics before you launch the campaign so you know what it will take. We actually have a measurement template that says “if you’re doing this, you should measure this.” For example, if you’re doing a Facebook event, you should measure # invited, # RSVPs (yes, no, maybe, not responded), # registrations sourced from Facebook, # wall posts.

    How is your nonprofit measuring integrating social media campaigns?