Can Stories Be Data? | Beth’s Blog

Can Stories Be Data?

Measurement

Flickr Photo by Caramel - Creatives Common Licensed

Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter?

There are lots of debates in the amazing world of measurement and learning.

The only valid data is quantitative data
Real time analysis is not useful because it is drive by analysis
Make data-driven decisions only, don’t trust your gut ever
Only measure impact, not process

And so it goes ……

Yesterday I came to the conclusion that it isn’t an either/or, but a both/and.  I shared this on Twitter and the conversation brought out some interesting points that I curated in “Storify,” a story telling tool.   (And, had to laugh at the geeky recursiveness of creating a story out of qualitative data from Twitter and other sources – and then adding it to my curated nonprofit measurement collection on scoop.it)

The network on Twitter responded:

  • If you’re overly emotional and reject quantitative data or unemotional and reject lived experience, you are less effective from Seth Godin shared by WSUCCSR
  • It’s fitness for purpose and don’t forget a lot of quantitative data is rubbish from Dunc-Rintoul
  • Will we ever move away from the either/or mindset from Jdeancoffey
  • When the issue of interest moves from measurement for accountability/compliance to#evaluation for learning from Karcsig

Perhaps we are confusing qualitative data with gut decisions?

I like to use a combined method.  I may start with numbers, but the process of collecting anecdotal information or stories in a structured way from your audience/stakeholders can help you generate insights about what those numbers actually mean.  Or as @orgnet says, “Turning data into stories is the real trick.”

Another way is to let qualitative data inform a hypothesis that you go on to test with numerical data.   Using unstructured qualitative data alone is pretty stupid unless you bring intelligence analysis, link to a quantitative analysis and link it back to decisions.

This leads to another question:  How can you transform anecdotes into useful data?

Last week at the Packard Foundation,  I participated in a conversation with Peter Laugharn, the Executive Director of Firelight Foundation about participatory learning agendas.   Some of the techniques include analyzing narrative data in a structured way to test and refine a theory of change.    This requires creating a culture of curiosity, a norm of continuous improvement, and leadership commitment — all hallmarks of a “data-driven culture”  whether the data be numbers or stories.

As part of work as Visiting Scholar at the Packard Foundation this year,  I’m facilitating a number of peer learning exchanges on measurement with grantees, including Global Giving and learned about their storytelling project which is an attempt to measure the effectiveness of how dollars raised are being used to create social change.

This approach is not about counting dollar signs, but about feedback loops which is discussed in  Michael Quinn Patton’s new book on Developmental Evaluation.  Storytelling Project is an experiment in collecting community feedback.   They started by recording thousands of stories or 19,181 stories to be exact told by people from areas where GlobalGiving partners work.

But how do deal with that much  data?   How do you make sense?  First, with discipline and structure.  Here’s their storytelling collection tool.      They’re using software – called sensemaker – to see patterns in the data that is far too much for human brains to summarize.  The synthesis – in the form of a presentation is shared with partners and more broadly.

The above shows the methods.   Now, if you’ve read this far and you work in a small nonprofit, you’re probably feeling overwhelmed.    I am too, just a little.   It made we wonder about how one can create a small scale, manageable method for a small organization to collect stories and use them as a feedback loop – AND part of their communications/content strategy.

The beauty of this — is that the stories work well as micro content and as an opportunity for engagement?

What do you think?  I’m noodling and it gave me an excuse to play with storify as a sense-making tool.

Is your organization collecting stories or anecdotal data in a disciplined way that helps you with your content strategy?   Is your organization leveraging this content asset further by using it as a continuous learning tool or marrying it to better understand your quantitative data?

 

31 Responses

  1. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  2. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  3. Marc Maxson says:

    To see how an “ordinary NGO” does all this sensemaking without the fancy software, or complex M&E design, or frameworks, read about Francis Guchuki of TYSA in Kenya:

    http://chewychunks.wordpress.com/2011/05/31/tysa/

    They were one of our earliest adopters (GG) because their community elders were already logging “significant” domestic events in little black books that were later mined for meaning. This ain’t brain surgery; it’s a culture of listening we’re trying to promote.

    Caveat: I’m a neuroscientist, so I actually know a little about brain surgery :) and can say that.

  4. Debra Durham says:

    A very informative and creative post. And, a great example of Storify, which I have only just discovered myself. Looks like there are some other great tools I need to check out, too. Thank you for the helpful suggestions.

  5. Thank you Beth for this important intervention into the data-driven conversation. In my experience in the world of educational policy, many on-the-ground experts are advocating a move to “data-informed” practice, feeling that data-driven processes can, and often have, steamrolled right over the substantial value of lived experience. The value of individual voices to enrich or inform quantitative approached cannot be underestimated.

    And for those who decry the subjectivity of qualitative data, I would heartily recommend that they have a few long conversations with those who design & analyze quantitative data. Most of us will tell you that quantitative data and its analyses are frequently subject to a similar level of interpretation, both in terms of the questions that are asked and the trends that are highlighted.

  6. Beth Kanter says:

    Alice: I’ve been writing about “data-driven” culture as part of the book I’m working on. And it is in the tag-line – “Measuring the Networked Nonprofit: Using Data To Drive Social Change” or something like that. I did a whole deep dive into what do we mean by data-driven culture – and I was using the term to refer to an organization that uses measurement to collect data and learn from – not just count stuff. Now, I’m thinking I need a new term .. “Curiosity-driven culture” What you think?

  7. Irene Guijt says:

    Data is defined by the Wikipedia
    1. Facts and statistics collected together for reference or analysis.
    2. Things known or assumed as facts, making the basis of reasoning or calculation.
    So I am not clear why stories cannot be data. If stories are qualitative data (though all stories contain ‘things known or assumed as facts’), then these can be quantified as is very evident from the GlobalGiving work. When I initiated discussions on this in 2009 with Rockefeller and Globalgiving became keen to try out the SenseMaker based work, I focused on the power of explicitly combining people’s lived experiences in the form of contextualised stories and the ability to sense patterns with the power of statistical tools.
    In other work I am doing using SenseMaker with the water sector in Ghana and Uganda, we used the patterns to drill down to interesting story clusters which were then debated with national and district level water professionals. These stories, quantified and contextualised, led to new ideas for action needed in the water sector and confirmed some existing activities.
    What is often more difficult is creating a story out of purely quantitative data, as we often don’t have enough context to make useful inferences or are tempted to make ones that seem rock solid simply because ‘the numbers prove it’.

  8. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  9. Hi Beth,
    Thanks for the link to createquity. I think the role of data in grantmaking is also an incredibly interesting conversation. In particular I’m struck by the shorter and shorter funding cycles of many granting institutions combined with the drive for more measurable outcomes. I’ve been really honored to work recently with http://www.mdcinc.org, which is working diligently to nurture systemic change that can alleviate institutional poverty. Important work? No doubt. Work that’s easy to quantify at the end of a 12-month grant cycle? Undoubtedly not.

    I love that term – curiosity-driven culture – because I think it gets to the heart of the intention for the use of data, which is critical. Are we compiling data/evidence to drive a change that, in at least some cases, may be predetermined or at least preferred (which will of course affect the research design and analysis)? Or are we undertaking an effort to genuinely better understand a central question or complex issue? In other words, is it really curiosity or evidence-gathering at play?

    I do want to just be clear that there are a tremendous amount of organizations, that regardless of terminology, are motivated by using data to better understand some of the most intractable social issues. They should be applauded for this work. But like anything else, not all data is created equal, so as data consumers, we need to be aware.

  10. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  11. christina says:

    Storify’s metrics appear just to be measuring on clickthrough.

    Aside from mapping URLs shortened and used in Storify, any suggestions for enhancing metrics on Storify stories?

  12. marnie webb says:

    I very much agree with the both/and approach to data collection and sense making. The quantitative bits provide a validity to our stories. A way for us to understand how reflective those stories are of our communitites and the issues in them. And the stories help us understand what the data means, how it impacts lives. How it plays out. I think they require each other for us to have well rounded pictures of ourselves, necessary change, and the impact of actions.

    And, as a side note, I’m in love with Storify.

  13. Beth says:

    Marnie: Thanks for your two cents here. I am wondering whether the story sensemaking approach really lends itself to evaluating networks.

    Anyway, how are you using storify?

  14. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  15. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  16. Lysh says:

    Hi Beth, thanks for this post. I firmly believe that quant and qual are both useful in painting pictures of supporter data. Your post sparked some blogging of my own and I’m suggesting – after reading your post – that all data analysts should be renamed storytellers!

  17. Kevin Gamble says:

    Hi Beth,

    We’e been using the Cognitive-Edge methods with great success. We’ve done a couple of studies, and are just launching our third on social media use in the enterprise. I’d be happy to talk with you about it sometime. It’s very good stuff!

    Kevin

  18. Beth Kanter says:

    Kevin: I was just thinking about you when I saw that Dave Snowdon had retweeted the link and I remembered that great Thai dinner we all had in Cambridge after your training on the software. I’m going to follow up …

  19. christina says:

    I’m seeing a large discrepancy between the Twitter clickthroughs to Storify and the Storify stat. Are others experiencing this?

  20. marnie webb says:

    I think that stories have to be a part of evaluating impact and understanding communities and change in general. And I think often stories can give us clues as to what we should actually be evaluating for (that’s certainly what I’ve found to be true as we look to understand the impact and potential of our various networks).

    I like storify because of the way you can collect and then organize the stories and add context. I love that actually. I find many of the curation tools to be heavier on the collection side and less on the curation side. so mostly my playing with it has been meta but i love what i see.

  21. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? [...]

  22. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  23. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  24. [...] Can Stories Be Data? Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter? Source: http://www.bethkanter.org [...]

  25. [...] Your numerical data can tell stories, but can stories be data that leads to continuous improvement?   Do numbers only matter?  [...]

  26. [...] Your numerical data can tell stories, but can stories be data that leads to continuous improvement? Do numbers only matter?  [...]

  27. [...] to experiment with the idea of how storytellers are using data in digital stories. I read a few interesting blog posts about how storytellers are not just including data in their stories, but exploring how the data [...]