How Can Nonprofits Switch to a Data-Informed Culture? | Beth’s Blog

How Can Nonprofits Switch to a Data-Informed Culture?

Capacity, Measurement, Organizational Culture

Example of A/B Testing Results

I’ve been reflecting on why some nonprofits do a better job of  measurement and learning, while others do not.  What is the difference?  It comes down to organizational culture.   The nonprofits that embrace measurement have a data-driven culture.  That is they make decisions based on meaningful data, rather than solely by gut.

Not all nonprofits are born with the spreadsheet gene.    And it isn’t simply a technical problem that can be solved through training or purchasing analytics software.   The  challenge has to do with organizational beliefs and work styles.    Whether it be a widely held belief that measurement practice is not worth investing resources.  Or a practice that swings the other away where there is an excessive investment in collecting gobbly gook data to appease a funder.

What is needed for  nonprofit organizations to make this shift?

The Evolutionary Stages of  A Data-Driven Culture

It is helpful to look at making the switch  as an evolutionary process.   In the end, it comes down to leadership.

Dormant: At this stage, the organization does not know where to start.  Does data collection may occur from time-to-time, but not formal reporting.   There are no systems in place, no dashboards or simple collection method.  Staff is often overwhelmed by thought of measurement and the task falls to the bottom of the to do list.     Or there is an emphasis on collecting lots and lots of data, but does not relate it to decision-making.  There not is a reflection process for analyzing success or failure to take into next use or campaign.

Testing and Coordinating: At this stage, the organization is regularly collecting data but in a bunch of different spreadsheets and collected by different people or departments.  Data is focused on the metrics that are specific to social media channel.   It is used to  improve  content, messaging, and engagement on specific channels.  Social media data is not linked to higher level organizational results or mission-driven goals across programs.   Discussions on how to improve results are rarely part of staff meetings.

Scaling and Institutionalizing: Has an organization wide system and dashboard for collecting measurement data that is shared with different departments.    Has different views or level of detail for senior leaders,  implementors, and different departments.    Holds weekly campaign check-ins to evaluate what’s working and what’s not across communications channels, as well as, any specific social media feedback received that would help shape our future campaigns or social media use.   Monitors feedback from target audience in real time but balances with trend or survey data.     Documents quantitative results against goals and value when asked by senior leadership.   Works with measurement consultants or specialists to improve skills and capacity.   Provides training and professional development for staff to learn how to use measurement tools.

Empowering: Sets organization wide key results areas and key performance indicators that are used across programs.   Has a staff position responsible for stewarding organization’s data, but staff are empowered to check and apply their own data.    In addition to weekly check-ins, the organizational dashboard includes  key performance metrics related to goals as well as more detailed metrics.  The organizational dashboard is shared across departments and there is a process for analyzing, discussing, and applying results.  They use data visualization techniques to report the data analysis but also to reflect on best practices culled from the data.

There is a regular report to senior leadership which details high level successes, challenges, and recommendations for moving forward.    Staff performance reviews incorporate how well the organization is doing on KPIs.  Works with measurement consultants or specialists to improve skills and capacity or to commission independent study and provides training and professional development for staff.    Celebrates successes by sharing measurement data across the organization.

DoSomething.Org: A Data-Driven Nonprofit In Action

DoSomething.Org is most definitely moving into the “Empowering Stage”  and are leaders in the non-profit world for exhibiting the characteristics and work habits of a data-driven organization.    Look at their approach to social media measurement in this terrific slide show by George Weiner, CTO, at DoSomething.Org called “What Does The Data Say.”   Despite being a relatively small nonprofit, they have a “Data Analyst” on staff, Bob Filbin.      What makes an organization to make this kind of investment in being data-driven?

It has to do with leadership.   Their Board, which is dominated by leaders in the tech field including Reid Hoffman, co-founder of Linked-In, and Raj Kapoor, co-founder of Snapfish, are all staunchly behind the philosophy of  ”The future of the web is data.”

The board supports the organization’s orientation towards using tech and data to realize its mission.   CEO  Nancy Lublin was  the driving force for hiring a data analyst and leading the charge for DoSomthing.Org to become a poster child for a data-driven nonprofit.

So, what does a data analyst do at a nonprofit?  It is more than hiring someone who knows how to program formulas in Excel spreadsheets.  Bob’s  job is to make sure that  departmental and overall organizational goals are aligned, and that social media data are seamlessly integrated into achieving their  organizational key results.

Bob’s responsibility is less to provide fish to staff, but more to teach them how to fish.   “My goal is to make sure that every person on staff has access to the data they need in order to create actionable changes in the way they do their programs. Ideally, each person will receive the data they need with automated dashboards that have different levels of detail and ladder up to organizational results.”

One of the biggest barriers in nonprofits for staff is finding time to devote to rigor and discipline measurement.  The time to collect data, the time to analyze, and the time to action on it.   Bob concurs.  ”DoSomething.Org understands the value of data-driven social change and has backed that up by creating a “data team” of three staff people.  For the past month-and-a-half, I’ve been working organizing our data collection, storage, analysis, and dissemination efforts.  Unless someone is put in charge of data, and it’s a key part of their job description, accelerating along the path towards flying is going to be hard, if not impossible.”

Bob points out the secret is to not to collect more data, but smarter data.   He says, “Just in case data collection can get in the way of achieving goals because it is wasted energy and time.  I am working with each department to make sure departmental  all data collection supports decision-making. ”

Do.Something is integrates critical metrics from social media,  e-mail, SMS, and Web.    They don’t just count the data, they use it to improve their tactics.    Says Bob, “DoSomething.Org uses A/B testing, where people can be randomly assigned to get different messages simultaneously. ”  This fall, DoSomething.org will start a push to acquire members via mobile phone, and A/B testing will be a crucial part of figuring out how to keep those new members engaged.  (The graph above is an example.)

Bob likes to quote Hal Varian, Chief Economist at Google,     “I keep saying that the sexy job in the next 10 years will be statisticians,” to others on the DoSomething.Org staff.  He thinks the business world is moving in the direction of more data analysis. “With the advent of social technology, we are facing an avalanche of data. The goal is to be able to sift through it, and find the diamonds in the rough on how to improve organizational effectiveness. That’s where statisticians, or data analysts, come in.   Non-profits know this is true, but the problem is investing in the resources needed to become a data-driven non-profits.”

Bob also believes that part of the problem moving away from making decisions by “gut” feelings, or intuition.  Bob says, “The data should tell us whether or not the program is effective.”

To make the shift, Bob suggests using small wins and share an example from an analysis on Facebook Ads for an event sign up.  ”We discovered the conversion rate was very low because we directed people to an external site (our web site) versus a sign up on Facebook.”   This insight will help use Facebook ads more effectively the next time around.”

Bob also talks about how to overcome resistance on staff to using data for decisions.  ”Your reports should be presented in a way that seeks to avoid bruised egos. Rather than bringing a number to a meeting, people should be reviewing their own statistics and data.     This is part of what I am doing at Do Something – closing the data loop. Making sure each department can access its data to answer their questions.”

Tips for Making the Switch

Culture is an organization’s operating patterns of behavior, activities, and attitudes, influenced by a shared set of values and beliefs that characterize the way people work together.  Changing a nonprofit culture isn’t as simple as
identifying the new ways of working you want to see or articulating a new set of beliefs and values associated with them.   Most people won’t change their behaviors until they observe the role models in their organization acting
differently as DoSomething.Org has done.   Also, when new behavior is positively recognized and rewarded — job promotions or some praise from the top of the organization –  change begins to happen.

1. Start at the top. Does your Executive Director know where the organization stands?  Educate through examples – showing how adding a data-driven approach to your social media can avoid ineffective campaigns and increase audience satisfaction.  The organization’s leadership needs to model and encourage a data-driven approach.

2.  Make the case to improve your measurement practice. The only way to evolve is through implementing a series of social media measurement pilots and small data wins.        Keep the end in mind when agreeing to how experiments will be structured, run, and measured.

3. Think big, but take baby steps. Start with looking at Key Result areas and key performance indicators, but since these may outcomes deal with long-term changes, you can’t get there overnight.  Keep the steps in the plan small and manageable.   Use measurement pilots.

4.  Share stories: Celebrate every bar graph that leads to a program or campaign victory.    Share it at staff meetings.    Also circulate stories about other nonprofits that have  become data-driven success stories.

Does your nonprofit have a data-driven culture?  How are you making the shift?  Where does social media measurement fall in that mix?

I’m working on a book with KD Paine about social media, networked nonprofits, and measurement.    Have a story to share?  Let me know in the comments.  You could be in the book!

19 Responses

  1. Thanks for the tips to get started. I think that’s often the hardest part. Another fear of nonprofits may be that it will cost a lot to collect data, and then, they may have to spend even more to do something about it – even if it’s cost effective in the long run.

  2. Joe Klem says:

    Another element I didn’t see above, and would be curious to get Beth’s take on it: nonprofits rely heavily on relationships — whether it’s with donors, partners, or other stakeholders. And sometimes the data may point you in one direction, but the valued stakeholder wants to go in a different direction. I’d venture that these relationships weigh more heavily in the nonprofit space than in the private/corporate world. Thoughts? Is there a stage of maturity where the org sheds this influence?

  3. Jay says:

    Great Post Beth!

  4. Geri Stengel says:

    The key is getting staff to view data as a tool that helps them do their jobs better. It’s not about numbers; it’s about continuous improvement and more social impact.

  5. Event360 says:

    Analytics are always important to track, but it can be hard to get people on board with tracking their work. Thanks for this advice on how to make this process a little easier.

  6. It is important also for an organization to be honest with itself about its abilities when looking at becoming more data centric. When the organization’s leader(s) come back from a conference about data, spouting off buzzwords and ready to spend money on information systems and services that they don’t understand, care must be taken to audit the organization’s resources and talent. $100,000.00 later and the boss is checking off an item on a checklist called “become data driven organization” and patting himself on the back while the systems and services that were purchased are barely being used. It really does come down to good management, even if that means being a good enough manager to admit that the organization does not have the right people or resources. If you don’t have anyone on staff that can manage this stuff, then by all means hire someone or outsource to someone who can get things set up for you. Have SOMEBODY at least THINKING about your data. You have to start somewhere.

  7. Holly Ross says:

    Hi Beth – love this post. As you know, we love data at NTEN and are really focused on helping nonprofits make this exact culture shift right now. That said, I do want to point out one bit that we find ourselves flubbing from time to time. We’re a pretty data-driven organization ourselves, and we find that from time to time we end up seeing a CAUSAL effect in the data, when there is only a CORRELATION. In other words, sometimes we think that factor X is causing the bump in the numbers for Y, when it turns out it’s only a coincidence. Of course, because we love numbers, when we switch up our strategy to address the issue, we end up seeing that faactor X is not, in fact, the cause pretty quickly. But it’s something I want to caution orgs about. You can’t just measure once. You have to measure over the long haul to be able to use the data well.

  8. Beth says:

    Holly: Great points and thanks for sharing that insight!

  9. Beth says:

    Joe: I’ve been thinking about your question a lot – and I think there is a delicate balance of valuing both intangibles – like relationships and financial outcomes.

    See this post: http://www.bethkanter.org/sm-toc/

  10. Mario Morino says:

    Thank you for an insightful post that takes on the sensitive topic of “being data-driven” for nonprofits and establishes at the outset that “it comes down to organizational culture.” Laying out the evolutionary stages of organizations along a “data-driven” continuum provides an important context and guide for those engaged in outcomes, performance management, accountability reporting, etc. And, incorporating DoSomething.Org’s experiences provides tangible examples and insights that will help people apply it to their organizations and roles. Well done!

  11. Milan Andric says:

    This is great discussion! While your typical non-profit, or company for that matter, may not have resources for a data analyst, everyone should be thinking about how to use measurement and data to help direct their role. Also it would be good to find tools that allow people to take advantage of measurement platforms and services that are available.. Each team should be empowered to take on the responsibility to measure progress. Many times these tools will be different for each group, and the first step is motivation and research. If we all agree the future is data then tools are being made as we speak… and many of our processes are the same. A productive direction from here might be a list of teams within an organization and what tools (or processes for that matter) are pertinent to their self measurement.

  12. min888 says:

    Many thanks for this valuable piece of writing. I actually loved going through it and will definitely talk about it with friends.

  13. Peter Gulka says:

    We’re heading down this road at my current employer. What is missing for me at this point is the storage mechanism. Once I start gathering all this wonderful data, where do I put it?

    Spreadsheets?
    Access?
    Salesforce?
    Raiser’s Edge?

    Pete

  14. [...] Kanter recently wrote a fantastic blog post on “How Can Nonprofits Switch to a Data-Driven Culture?” It is worth a read if you are struggling with data management and reporting. Her article is [...]

  15. [...] Kanter recently wrote a fantastic blog post on “How Can Nonprofits Switch to a Data-Driven Culture?” It is worth a read if you are struggling with data management and reporting. Her article is a great [...]

  16. I found this post from another link. It’s not social media data, but I’m working with an organization right now that has a Quality Improvement Department, the function of which is to collect data about programs and their outcomes, unmet needs, and other key metrics, and then to share those data with program staff, grant writers, and leadership in ways that can facilitate effective decision making. Again, even though they are very different kinds of data, the elements of a ‘data-driven culture’ that you identify really resonate. I, too, appreciate the ‘how to start’ advice. And I can’t wait for the new book!

  17. [...] issue is making sense of it all. And the first step forward is a mind shift toward becoming a data-informed nonprofit [...]

  18. [...] I've been reflecting on why some nonprofits do a better job of measurement and learning, while others do not. What is the difference? It comes down to organizational culture.  [...]

  19. assume workforce provides competing. “We may placement the particular assist a lot of issues, inches they defined, “staff pays for boosting they will Received Brees Shirt. My business is included in many of the commerce discussions, When i fully generate.