The Outcome of Our Outcomes | Beth's Blog

The Outcome of Our Outcomes

Guest Post, Reflection

Flickr Photo By Mastery of Maps

Note from Beth: I’m hosting a small army of guest bloggers, grantmakers, who are attending the  GeoFunders National Conference taking place this week in Seattle.   The GEO community is united by a common drive to challenge the norm in pursuit of better results. GEO’s 2012 National Conference  shares a range of perspectives and new ideas for smarter grantmaking that leads to better results and presents opportunities for participants to learn from the wisdom and experience of their peers.     If you’re not attending and curious what funders are learning,  you’ll have an opportunity to read some of the ideas and questions being discussed right here on this blog.

The Outcome of Our Outcomes – guest post by Victoria Vrana

It’s almost Spring and a thousand theories of change are blooming like the cherry trees in Washington, DC.  Outcomes, indicators, metrics, logic models, flow charts galore – measurement and evaluation are in season these days.  But during the session on Learning in Public yesterday, with Beth Kanter, Co-Author of Networked Nonprofit, Jared Raynor from the TCC Group and Kathy Reich, from the David and Lucile Packard Foundation, I was reminded that all measurement needs to start with the most fundamental question – why measure?

The Packard Foundation, true to the nature of its brave and innovative Learning in Public experiment, generously shared with us some of the lessons it learned by opening up the measurement and evaluation effort of its Organizational Effectiveness program to…well… the public.  One of the biggest lessons it learned was the need to set sharp and clear goals at the start.

Measurement springs from many different motivations.  It is critical you know the why before you start designing, doing, and paying for the what. Are you measuring to test a hypothesis?  Is your measurement approach a way to manage your grants and portfolio?  Is the evaluation of your program something management needs to assess the performance of your organization as a whole?  Do you hope to fundraise or garner more support by measuring? Are you required to measure?  Are you creating your measurement approach/designing your evaluation, or did someone prescribe the approach you now need to implement? Do you hope to assess and improve your performance? What do you hope to learn? The most important question to ask yourself before you start: If measurement shows you are not on track to achieve your goals, are you willing to change course or establish new goals altogether?

Goal setting is often rushed.  It’s a preamble to an RFP.  A throat clearing at the beginning of the meeting.  With all the new and fashionable ways to tackle evaluation, we’re frequently leaping to the how without asking ourselves the hard questions that make such a difference to the design of our efforts, the scope of the work, and the commitment to how the results will be used. The goals need to be front and center through the entire process, particularly when the measurement approach is more new or participatory and is designed to evolve over time.

Kathy Reich also shared with us how they learned engaging the “public” creates some energy and excitement and benefits for others, but their target audience for participation is actually “first tier stakeholders,” starting with grantees. Knowing who you want to involve and reach is critical. Data, reports, and dashboards are all well and good, but if they aren’t tailored to the needs of the core audience, the information will never be accessed or used going forward.

The Packard Foundation did find significant value in Learning in Public and has launched an effort to refresh the strategy for its Organizational Effectiveness Program in public, with crisp goals and a clear audience.  I’m looking forward to being one of the public who gets to benefit from their courage and transparency.

We are about to embark on a measurement approach process at the Charitable Sector Support initiative of the Bill & Melinda Gates Foundation and before I take another step, I’m going to dust off those goals we set in the early days just to make sure we’re still on track.

If you’re currently involved in designing or implementing an evaluation or performance measurement effort, what are your goals? Who is your audience?

Victoria Vrana

Victoria Vrana is a Senior Program Officer on the Charitable Sector Support team of the Bill & Melinda Gates Foundation.  Disclaimer: The postings on this site are my own and do not represent the positions, strategies or opinions of the Bill & Melinda Gates Foundation

2 Responses

  1. Jusna Perrin says:

    I am currently designing and implementing an evaluation system for our 2-yr old organization. The program itself trains high school struggling readers to tutor elementary school students who are also struggling readers.

    The goals of my effort is to measure not only reading levels of both the elementary and high school students, but student efficacy and engagement, program satisfaction (of our schools, parents and students), vocational aspirations (of our high school students), school attachment and positive relationships, session attendance, as well as school attendance, retention and promotion.

    Our audience in these efforts include the students, parents, program staff, board of directors and funders. In all evaluation efforts, our organization aims to improve our organization.

Leave a Reply