Social Media Measurement and Learning Analytics: How Do I Love Thee, Let Me Count the Ways | Beth’s Blog

Social Media Measurement and Learning Analytics: How Do I Love Thee, Let Me Count the Ways

Measurement, Training Design

Yesterday, I had the honor of kicking off Compasspoint’s Webinar Week, with a webinar on social media measurement 101 as part of my work as Visiting Scholar at the David and Lucile Packard Foundation.     To mark Valentine’s Day, I took some inspiration from Elizabeth Barrett Browning and the vintage Valentine’s Day Cards pinterest board to create a presentation about social media measurement love.   I tested out the five phases of falling in love with measurement.    Given the topic was measurement,  I couldn’t help but go a little meta and play with incorporating learning analytics into the instruction.    This blog post shares some insights about those two somewhat disconnected ideas.

The Five Stages of  Social Media Measurement Love


It is always important to look at the challenges of adoption of improving organizational processes or individual skills, especially if they are new.    It should always accompany the tips and how tos.   That’s why I always start a webinar with a framework for learners to assess where they are and what change is needed (if anything) to embrace a new way of doing something.     I presented the five stages of falling in love with measurement.  These are:

1.  Denial: You can’t measure social media!
2.  Fear: What if we find out our social media didn’t perform well!
3.  Confusion: I don’t know what or how to measure social media.
4.  Delight: Check out these charts and graphs!
5.  Measurement Love: Connecting measurement to decision-making and getting better results

I did a quick user poll for folks to self-asses where they are and then we had a quick discussion in the chat to unpack what each stage looks like and what’s needed to get to the next stage.      I was surprised to hear from participants that while they have the practice of measuring other programs, they don’t know how to approach measuring social media.      The “fear” stage seems to also go hand in hand with fearing social media in general. Said one participant,  “We are in the fear stage because our senior leaders fear social media and we’re not sure we get results with social media.”     Many admitted to being in the “delight” stage – that collect data but don’t know how to make sense of it.    Those that found themselves at measurement love – using data for decision-making said that it goes hand-in-hand with their organization’s learning culture.

Learning Analytics

I came across the term “Learning Analytics” in an NPR story about the effectiveness of college lectures and ways to make learning more interactive.      There are different definitions.   Elearning space defines it as: “Use of intelligent data, learner-produced data, and analysis models to discover information and social connections, and to predict and advise on learning.” I took the very simple approach of thinking of it as a data that can help you see the correlation between learning activities and learning outcomes.

The outcome for this webinar was:   To leave the webinar ready to take one small step towards improving social media measurement practice.

To do that, participants have to be able to implement that step without being encumbered by the denial, fear, confusion, or delight stages.      I took several polls throughout the webinar asking them about their relationship with measurement.  Specifically, I wanted to see if the needle moved from hate to love.      If I look at the pre and post tests, the “love” measurement increased.

 

I also used this poll after different sections of content.     The polling after the section of content about how to make goals measurable showed that the needled moved slightly in the other direction, towards hating measurement.  A follow up question revealed that a big challenge is making goals measurable.       This gives me some more insight about how to make teaching this concept more accessible.

This was just a first experiment in trying to use the concept of learning analytics to help improve instructional design.    It is something I want to explore in more depth – I know that I’m probably not asking the right questions or perhaps not stating them in the right way.    The next step is to formulate questions that relate to the content not learner attitudes.

Bonus:  Spreadsheet Appreciation

One of the most important measurement tools for nonprofits is the spreadsheet.     In this post, Mashable reviews five free awesome social media spreadsheets.   I tested each one.   The one I found most useful was #3 because it could be an easy way to compare or benchmark with other organizations.  However, the use of a pie chart is a little meaningless – I would change to a bar chart.

Do you think that understanding learner’s attitudes and their organizational context is important when designing instruction on new ways for nonprofits to work?    Do you think using learning analytics is of value to helping you improve instructional design?     Have you used learning analytics?   Please leave a comment.

 

 

 




14 Responses

  1. Taryn says:

    This was extremely helpful – thank you!! One of the best takeaways was learning that 52% of people are just as confused as I am about measurement 😉

  2. Beth – simply love your article and the graph on five stages of social media measurement love. I work in the Indian market and I see the same stages with our clients. However I have to still make them reach ‘measurement love’.

  3. Beth says:

    @neelima: Do you work with nonprofits or for-profits? Where would say most are?

  4. Jim Sterne says:

    Social Media Metrics? Why, I could write a book!
    Oh wait, Beth just delivered everything I wrote about in one, short blog post. Guess I don’t need to write the second edition now… Thanks Beth!

  5. Beth says:

    Jim: Wow, a second edition is coming! Yeah! Can’t wait.

    Are you writing about learning analytics? and Happy Valentine’s Day!

  6. Jim Sterne says:

    No, no —- I do NOT have to write a second one! Besides, John Lovett did a great job here: http://www.amazon.com/Social-Media-Metrics-Secrets-Lovett/dp/0470936274

  7. Beth Kanter says:

    yes, John’s book was awesome …. and the analytics exchange with Eric Petersen has helped so many nonprofits
    http://www.bethkanter.org/curator-metrics/

  8. […] background-position: 50% 0px; background-color:#222222; background-repeat : no-repeat; } http://www.bethkanter.org – Today, 4:09 […]

  9. Kelly Meeker says:

    I work in the learning technology sector, and I think the entire sector is split between two groups: 1) What’s measurement? and 2) I know I should be doing analytics but I don’t even know where to begin. I seriously doubt there’s a large population of folks who can enumerate how they use analytics thoughtfully.

    For too long the learning sector has been stuck in the same style of metrics that many NGOs have – “we’ve provided X resources” or “we’ve distributed Y widgets” – rather than measuring changed behaviors or outcomes. Identifying those desired outcomes and then translating the steps there into steps that can be measured is the needed skill.

  10. This is really interesting. I do think understanding learner attitudes is important to developing a measurement system that will work for the organization in question. If people don’t understand what they’re measuring, and why, it will be pretty difficult for them to read/understand the value of any metrics. And if it’s difficult, people don’t value it much. No one wants to look like they don’t understand stuff. So eyes glaze over, people nod their heads like they ‘get it’, and then they move on. The reports they receive go in one ear/out the other; therefor, the measurements don’t really inform strategy moving forward. Lots of food for thought here. Thanks!

  11. Beth says:

    Hi Kelly: Thanks for your thoughts about comparing the learning technology sector to nonprofit sector. Very interesting. Do you do measurement as part of your work?

  12. Kelly Meeker says:

    Beth, I work with our customers and community members on developing analytics to measure the effectiveness of their learning support programs. One idea I’m experimenting with specifically is connecting the Theory of Change model (http://www.theoryofchange.org/) to learning measurement. I’ve got an early exploratory blog post in the works – will share when published!

  13. […] background-position: 50% 0px; background-color:#222222; background-repeat : no-repeat; } http://www.bethkanter.org (via @csjolley) – Today, 7:21 […]

  14. Greg Jonason says:

    Also interested in Learning Analytics. Heard of these?
    http://www.noodle.org/sitemap?sitemap_vertical=TC

    Brightstorm
    Grockit
    Junyo
    Knewton
    Snapwiz
    Prepme
    ePrep
    Prephub

    Another thing, something strange at the bottom of your blog in Firefox on Mac (v10.0.2):

    ocial Media Measurement and Learning Analytics: How Do I Love Thee, Let Me Count the Ways | KCKCC Online Ed News | Scoop.it says:
    February 22, 2012 at 7:21 am

    […] background-position: 50% 0px; background-color:#222222; background-repeat : no-repeat; } http://www.bethkanter.org (via @csjolley) – Today, 7:21 […]