Data-Informed, Not Data-Driven | Beth’s Blog

Data-Informed, Not Data-Driven

Measurement, Organizational Culture

Flickr Photo by epsos.de

I saw an interesting post from Eric Petersen called “The Myth of Data-Driven Business” by way of KD Paine’s blog.   It caught my eye because we’ve working on a chapter about  how nonprofits can make the shift to a “data-driven culture.

Holly Ross at NTEN and I chatted about the myth of data-driven nonprofits  and how this is slightly different with mission-driven organizations.    Some nonprofits consider themselves “data-driven” simply because they collect data (often at the request of funders), but they may not necessarily be collecting the right data!    Collecting data does not equal being data-driven.

Eric Peterson calls out a discussion on “What the C-suite should know about analytics” from Kishore Swaminathan, Accenture’s chief scientist:

“Data is a double-edged sword. When properly used, it can lead to sound and well-informed decisions. When improperly used, the same data can lead not only to poor decisions but to poor decisions made with high confidence that, in turn, could lead to actions that could be erroneous and expensive.”

Eric Petersen says that it requires  a balance, and  leaders should not become slaves to the numbers, but leave room for business knowledge and experience to help them understand and learn.   Eric says that the term being “driven” implies   — “going out of your way to devalue experience, ignore process, and eschew established governance in favor of a new, entirely metrics-powered approach towards decision making. ”

He suggests a new term, “data informed.”

I suggested the phrase  “curiosity-driven.”   That an organization knows the questions to right ask, how to collect the right data to answer those questions.  Above all the organization uses its collective wisdom to making decisions.   That the nonprofit doesn’t  just look at the pretty charts and graphics as if it they tea leaves, but brings multiple lens to understanding how to apply the data.   And, it is part of a process of learning and continuous improvement.  But, Eric’s “data informed” is much better.

What are the myths of being a  ”data-driven” nonprofit?    Is there a better term to describe making decisions based on combining data with organizational experience to learn?

Update:  Cross posted this on Google+ – Conversation Summary

Data-driven and data-informed are two different things and that it is better to be data-informed because you need knowledge to interpret the data.    Being data-driven can lead to a/b testing 40 different shades of blue.     Ian Thorpe from Unicef said:

“Decision makers to have better skills in using data before they are too driven by it. But I also have a theory (but no data to support it) that there are two ways to look at a problem – the first is to analyze data and draw conclusions – the job of an analyst or researcher, the second is to form your opinion by consulting with a diverse network of people whose judgement you trust (including but not limited to analysts) – the second is what most leaders do since they often don’t have the time or skills to do the data analysis themselves – and the social interactions are what helps make sense of the data. I think we actually need a bit of both sorts of decision making.”

Stephen Downes summary of the conversation here

 

8 Responses

  1. [...] Curiosity-Driven, Not Data-Driven Flickr Photo by epsos.de I saw an interesting post from Eric Petersen called “The Myth of Data-Driven Business” by way of KD Paine’s blog.   It caught my eye because we’ve working on a chapter about  how nonprofits can make the shift to a… Source: http://www.bethkanter.org [...]

  2. Beth, this is such an interesting topic. I would add one comment to the balance point raised by Ian. One of the challenges many non-profits face is to become more effective storytellers. A dependence on data can make that even more difficult than necessary and eliminate the human quotient from the decision making process. Ultimately data is critical to evaluate the emotional connection non-profits have made with people and their shared concerns and that order of priority is critical if non-profits, or any brand for that matter, hopes to use social media to build their community, fundraising or profile. The gut check, or rather heart check, are still critical tools in business and social media strategy and an obsession with data can overwhelm their importance and the benefits they offer.
    Thanks, Beth. simon

  3. Beth says:

    Simon, totally agree with you -did you sere “Can Stories Be Data?” http://www.bethkanter.org/stories-data/

  4. > What are the myths of being a ”data-driven” nonprofit?

    Either the promoters are not aware of research in the philosophy of science, or they are aware of it, and hoping staff and members are not.

    The research is simple: all data are “theory-laden”. Data is not value neutral. It is not the raw material used in the formation of practice and theory, it is inevitably the result of it.

    Some references:

    - Larry Laudan, Progress and its problems: Towards a theory of scientific growth. http://bit.ly/qJyTxd
    - Here’s a summary of Norwood Russell Hanson, whom I quote a lot: http://www.loyno.edu/~folse/Hanson.html
    - Stanford Encyclopedia of Science, Theory and Observation in Science http://plato.stanford.edu/entries/science-theory-observation/

    If the promoters are simply unaware of these (and numerous additional) works, then they may naively believe that data simply informs theory and practice, without incorporating some bias or prejudice of its own. In this case, the promoters would simply be wrong.

    Since the promoters of data-driven practice are intelligence and well-educated, it is likely they are aware of the theory-ladeness of data. Very likely, they are defining the data to be relied upon. If so, then they are using data to surreptitiously impose some perspective or world-view without discussion or dissent.

  5. Beth says:

    Stephen: Thanks for introducing me to the “theory-ladeness of data” or it sounds like lying with statistics?

    Note from email: Via Amy Lucky

    The book is The Pathfinder by Nicolas Lore. The chapter is “Making Decisions – A short course” (page 140).

    The punchline is relevant to organizations as well as individuals:
    * Make decisions deliberately, not by default (sounds obvious, but is so rare – because it is so hard.)
    * Use multiple information sources to inform decision making – rather than always using your usual go-to source (be that “data” or “gut” or something else)

    I am sure the same points can be found in organizational lit or biz lit, this just happens to be written in a way that resonated with me and the source that I have.

  6. Josh Dormont says:

    Well said, Beth. I particularly like the “Being data-driven can lead to a/b testing 40 different shades of blue.” This is one of the biggest lessons that folks like Bryan Eisenberg bring to the business of optimization and analysis. It’s not simply the ability to test and measure that we should relish, but rather the ability to focus in on the things that will lead to the greatest impact.

    This is always perceived as something that is much harder to do in non-profits because when we test, we often test with people’s lives to some degree. But being data-informed means being willing to push our thinking on the best way to deliver services, etc.

    My own thoughts on the value of data-informed testing on work within an organization: http://wp.me/p1OWHB-2K

  7. Great entry and great comments. Thanks for the links.

    I’ve been thinking about this recently. It seems that there’s almost a backlash–but not an irrational one–to the impact mania that’s come up with social enterprises realizing they need more than profit to track their social impact. I love the idea of a curiosity-driven model, of course within reason. But curiosity will to some extent check itself, because it will always be asking, “Does this even make sense? What else is out there, and what about this other perspective?”

    Curiosity is like a combination of learning and creativity. Love it.

  8. Asme y14.5 says:

    naturally like your website however you need to take a look at
    the spelling onn quite a few of your posts.Several of them aree rife with spelling problems
    and I to find it vsry bothersome to tsll thhe truth nevertheleess I
    will certainly come again again.

Leave a Reply