Nonprofits Collect Lots of Data, But Most Don’t Use It Says NTEN/Idealware Report | Beth’s Blog

Nonprofits Collect Lots of Data, But Most Don’t Use It Says NTEN/Idealware Report

Measurement

The State of Nonprofit Data report was released this week  (hat tip to Ted Fickes).   Idealware and NTEN prepared the report based on a  survey in April 2012 with nearly 400 nonprofit organizations about how they manage and use data.   The report found that nonprofits are either doing a lot with their metrics or not much at all.

 

These barriers that keep nonprofits from collecting data and identified in the report include:  data collection, expertise, time, and money.  These barriers are precisely why  I wanted to write  ”Measuring the Networked Nonprofit:  Using Data to Change the World” with KD Paine.    In Chapter 3 of our book, we talk about the skills and practices of a data-informed organization.   The reports shares the infographic below to illustrate the “Data Machine.”   The report notes that a “well-functioning nonprofit data decision-making process provides numerous opportunities for nonprofits to improve their programs and get better results.”   But far too often the processes of nonprofits did not turn out useful data or in some cases prevented it.

The report goes on to describe the type of data that nonprofits collect which include:  Financial and internal operations data; marketing, communications, and fundraising data and tracking programs and outcomes.  The report offers some insights about the state of data collection in nonprofits, especially with respect to tracking the impact of their programs:

When it comes to tracking the actual impact of their programs, we heard many possibilities for looking at the direct effect of programs—for instance, client satisfaction surveys, number of repeat clients, client recidivism, and the percent of issues resolved for clients, all of which can help to inform nonprofits’operations. many of the organizations in our focus groups wanted to move beyond this level to look at their actual impact in the community, but were struggling to do so. While many nonprofits look at metrics that would actually measure the long term outcomes of their work as the “holy grail” of data-based decision-making, it’s unclear that this level of measurement is practical or even possible for many nonprofits. many orgs need to either rely on metrics which are indirect measures of long term impact—like the number of repeat clients, client satisfaction, or percent of client issues resolved—or invest in expensive, long-term longitudinal research. for instance, one of the nonprofits in our focus group was interested in measuring to what degree they were improving high school students’ success in school by engaging them in creative writing. To directly measure a high-level impact like this, however, one would need years of high quality data summarizing success in school (grades and attendance, for example)and, ideally, a control group with very similar demographics and attributes. This kind of data would be difficult to get from schools, and the research design, data tracking and analysis would be time-consuming and expensive.

In our book,  ”Measuring the Networked Nonprofit” one of our key messages is not to start collecting data or think about tools until your organization has defined success.    We lay out a simple framework, “The 7 Steps of Measurement” that can be used to measurement type of nonprofit program.    So, it is eye-opening to see that between 25-42% of respondents have a plan for defining success and having a plan to collect data that will help determine if they are successful.

It is also interesting to see that between 26-50% of nonprofits surveyed use their data to make decisions.    Jocelyn Harman in her review of our book put it another way.  Nonprofits need to “loose their gut and get serious about measurement.”

The survey also provides some data to answer the question, “What is the funders role in supporting good measurement?”   The report identified challenges, including the varied funding and data collection requirements of different funders and reporting demands.   Finally, the report ends with some high level advice about how to get started with measurement and data collection.   These recommendations are very closely aligned with the in-depth and practical how to steps we outline in Measuring the Networked Nonprofit.

While this report focuses on the challenges, barriers, and problems that nonprofits face in using good measurement practices and collecting data,  a session at last weeks Independent Sector called Data 360 focused on the best practices.      The session was a poster session where about a dozen of the leading practitioners of being data-informed in the nonprofit sector lead small group discussions to share best practices.

 

While report reveals that the nonprofit has a lot of room for improvement in using measurement and data, there is a great opportunity for us to continue the dialogue and share best practices on sites like Markets for Good, these resources from IS Sector and, of course, at the NTC. But remember, data is only part of the puzzle -it’s how you make sense of it and apply to constantly improve your organization’s impact and results. This requires a balance of what I call “left brain” (number crunching) and “right brain” (creative thinking) that can lead to innovative approaches to solving the wicked problems that our social sector wants to solve.

6 Responses

  1. “The report found that nonprofits are either doing a lot with their metrics or not much at all.” This has been my experience with so many of my nonprofit clients. There’s a whole lot of data floating around organisations but they often don’t have a clue!

    Even data portability has caused to collect but never use data. The big database they spent hundreds of thousands of dollars getting ready is never used because they can’t get the data from one system to another. That’s probably a skill factor most of the time but why should it be?

    In my mind, just beginning with using small amounts of data and build on top of incremental internal capacity and competencies. Surely that’s a wise way of going about using data to become a data driven nonprofit.

  2. Beth Kanter says:

    Andrew: I think you are right on in your suggestion of taking baby steps with data collection. Skills are one issue, but organizational culture is another.

  3. Annaliese says:

    Thanks for posting about this, Beth, and for making the connections between what we found as barriers, and the practical steps your book puts out there that organizations can take to overcome these barriers.

    And I also agree with Andrew’s suggestion that starting small is a smart way to go. I think many nonprofit staff get overwhelmed by the amount of data out there and what it might take to put it to use. Starting with a specific, achievable plan for what an organization needs to “know” or “do” — before even thinking about the data — can help prevent the sense of defeat about having the time, money, and expertise to turn all of the data we have access to into outcomes.

    Beth is right: it’s about organizational culture.

  4. Mark Rubin says:

    Great mesage, but I think the problem is far broader. Governments don’t use data well, and corporations and individuals don’t either. Everyone can do a better job, but I do think to some degree the private and for profit sectors hold government and nonprofits to standards they don’t meet or even consider. (My pet peeve is govenrmental inefficiency. In my day job–busienss attorney–I see lots of inefficiency, often form people who lambast everyone else for costs, overhead and inefficiency.

    Some of the problem is math illiteracy, in addiiton to the problems you mentioned. Many people are simply ill-equipped to understand what they see.

  5. [...] investigação revela que as organizações sem fins lucrativos recolhem toneladas de dados, mas que não estão a [...]

  6. Terri says:

    One of the issues we are grappling with is how can we make more accurate revenue projections? Are there any best practices for nonprofits?

    Thanks!
    Terri