Methods and Metrics for Assessing Civic Tech | Beth’s Blog

Methods and Metrics for Assessing Civic Tech

Engagement, Guest Post, Measurement, Research Studies

Note from Beth: My colleagues at Network Impact have released an important report about measuring and evaluating “civic tech” based on their experience working with Knight Foundation grantees.     It goes beyond just analytics tools to exploring tools, techniques, and data for measuring deep civic engagement, open governance, and inclusive decision-making.   My colleague, Anne Whatley, wrote this summary of what you’ll find when you dig into these two stellar and highly practical reports and resource lists.

Methods and Metrics for Assessing Civic Tech, Guest post by Anne Whatley, Network Impact

Digital technologies are connecting residents in neighborhoods, catalyzing community discussions, changing the way governments and citizens interact, and making government more transparent. This growing field of “civic tech” is aiming to harness technology to spur citizen engagement, improve cities and make government more effective.  The Network Impact team spent 18 months consulting with civic tech thought leaders and offering technical assistance to Knight Foundation grantees focused on assessing the impact of their technologies. One thing that stood out in our conversations is that the civic tech field is ready for a deeper dive on outcomes. Designers of these civic tech platforms and apps are already measuring results, such as the number of users, and are very familiar with tools like Google Analytics. But these instruments don’t explore deeper changes like increases in civic engagement, more inclusive community decision-making, and open governance that may be occurring as a result of their initiatives.  A big topic of discussion as the field is maturing is how to track on-the-ground outcomes for people and communities, in addition to counting clicks, views, downloads and tweets.

Code for America, one of the organizations that we worked with, is already implementing outcome assessments. Code for America tracks their impact on open governance ecosystems by monitoring a range of results, including the number of apps created by fellows that are sustained by municipal administrations. In addition to tracking app deployments, Code for America has developed metrics that help them understand their impact on the governments they work with, the communities they engage, and the larger civic technology ecosystem. When we asked Code for America team to summarize the need they see in the field, they responded, “The civic tech field is growing rapidly. Demonstrations of what is possible are up and running in communities of every size across the United States. Now we need to find out not only what works, but what works best over time.”

A few lessons for assessing civic tech from our work include:

A Team Approach

It’s important to engage your full team, especially tech development staff, early in evaluation planning to ensure that systems are in place to capture the data you’ll need to track progress and assess results. For example, if location information will prove useful in tracking place-based impacts, best to build this metric into registration from the start, rather than try to capture the information later.

Hone in on your true scope of influence

Start by focusing your assessment on the outcomes that your app or platform is most directly designed to produce (such as increased trust between neighbors or resident contributions to a specific local decision-making process) rather than on very broad overarching goals like “strengthening participatory democracy.” Strong evidence of success in reaching a targeted objective will create a firmer foundation for future efforts and send a clearer signal to supporters, users and the field than a broader claim for which you have limited evidence.

Learn and borrow from others

Seeing how others are doing this work can help you develop or improve your assessment plan and lead to improved results. The guides we created based on our work with civic tech teams include many examples of outcome assessments linked to overarching objectives that include increasing civic engagement, promoting deliberative democracy, and supporting open governance.

Guides to Civic Tech Assessment

We summarized what we learned in two guides that are designed to help civic tech innovators monitor progress towards their intended outcomes and evaluate the impact of their efforts.

Our website also includes Additional Tools and Resources with links to more articles and useful references.  If you have any suggestions for inclusion please send us a note.

Anne Whatley is a consultant with Network Impact providing advice, research and tools to support social change networks, foundations and the emerging field of network builder

 

5 Responses

  1. […] you catch our guest post on Beth Kanter’s blog? Methodologies and Metrics for Civic Tech shares a few lessons learned working with the Knight Foundation’s Technology for Engagement […]

  2. […] Read More Share this:TwitterFacebookGoogleLike this:Like […]

  3. […] which civic tech solutions are actually creating change? Anne Whatley from Network Impact offers some tools for assessment in that […]

  4. Steven Clift says:

    As part of this review, E-Democracy.org received help with the design of our participant survey.

    We feel it is extremely important for organizations in the civic tech space to share and learn together. So to that end, here are our raw results: http://bit.ly/edemsurveyresults

    There is an opportunity for researchers and others to dig in more deeply with our large 1300 respondent sample from one focused region.