At OpenUp, we build tools for change. There is no creating tech for tech’s sake; instead, we believe there are ways to use tech (and create tech) to do good - but to do this, you have to approach with intent.
A strategy for trying to make sure you get tech to do what you want it to do is to measure the impact it is making - but how might an organisation go about doing this? Here is a quick FAQ to help introduce our approach to measuring the impact (and change) our tools bring to the world:
Measuring the impact of civic tech tools is relevant to both those inside and outside the organisation. There are three main reasons for doing it. Firstly, to inform long-term internal decisions (e.g. where should we spend our time) by understanding how much impact a tool has and how this compares to other tools. Secondly, to drive improvements in tools by understanding how the impact is being generated. Lastly, to improve communication of the impact of the tools with other partners. While it may not be easy, measuring impact is essential.
Impact measurement in the Civic Tech space needs to focus on changes in end outcomes and not just on traditional tech engagement metrics. Traditional short-term “online” measures of success (website hits or sign-ups) do not necessarily align with long-term “offline” measures of impact. Instead we must understand how tools are being used, and by who, to understand their impact.
Causality is hard to prove in the environments where Civic Tech projects operate. This is because the connection between a tool’s effects on its users and on society requires a series of steps between participation and the end outcome which are often unclear and occur over a long period of time. We often care about the impact a tool is having on a system, which is likely complicated to understand.
There are typically limited organisational resources for impact measurement. Funding is often secured for specific projects and civic tech organisations typically have few monitoring and evaluation focused resources which means there can be limited opportunity to evaluate the impacts of civic tech tools.
To understand the impact of a tool you need to be able to measure three things: who is it using and what for, how many of them are using it and how effective the tool is. Each of these components is essential and each likely requires a different approach and data source. Any process to measure the impact of a Civic Tech tool will require assumptions and be imperfect, however we believe this framework offers potential.
At OpenUp we have implemented the following steps on some of our projects in an attempt to measure impact:
As we’ve begun experimenting with this approach to impact measurement we’ve begun to learn things about both the impact our tools generate as well as the process through which we can measure this.
What we’ve learnt about impact:
What we’ve learnt about process:
At OpenUp, we build tools for change. There is no creating tech for tech’s sake; instead, we believe there are ways to use tech (and create tech) to do good - but to do this, you have to approach with intent.
A strategy for trying to make sure you get tech to do what you want it to do is to measure the impact it is making - but how might an organisation go about doing this? Here is a quick FAQ to help introduce our approach to measuring the impact (and change) our tools bring to the world:
Measuring the impact of civic tech tools is relevant to both those inside and outside the organisation. There are three main reasons for doing it. Firstly, to inform long-term internal decisions (e.g. where should we spend our time) by understanding how much impact a tool has and how this compares to other tools. Secondly, to drive improvements in tools by understanding how the impact is being generated. Lastly, to improve communication of the impact of the tools with other partners. While it may not be easy, measuring impact is essential.
Impact measurement in the Civic Tech space needs to focus on changes in end outcomes and not just on traditional tech engagement metrics. Traditional short-term “online” measures of success (website hits or sign-ups) do not necessarily align with long-term “offline” measures of impact. Instead we must understand how tools are being used, and by who, to understand their impact.
Causality is hard to prove in the environments where Civic Tech projects operate. This is because the connection between a tool’s effects on its users and on society requires a series of steps between participation and the end outcome which are often unclear and occur over a long period of time. We often care about the impact a tool is having on a system, which is likely complicated to understand.
There are typically limited organisational resources for impact measurement. Funding is often secured for specific projects and civic tech organisations typically have few monitoring and evaluation focused resources which means there can be limited opportunity to evaluate the impacts of civic tech tools.
To understand the impact of a tool you need to be able to measure three things: who is it using and what for, how many of them are using it and how effective the tool is. Each of these components is essential and each likely requires a different approach and data source. Any process to measure the impact of a Civic Tech tool will require assumptions and be imperfect, however we believe this framework offers potential.
At OpenUp we have implemented the following steps on some of our projects in an attempt to measure impact:
As we’ve begun experimenting with this approach to impact measurement we’ve begun to learn things about both the impact our tools generate as well as the process through which we can measure this.
What we’ve learnt about impact:
What we’ve learnt about process:
