Guest Blog: Evaluation & Impact
Thanks to Ellie Cripps from University of Bristol's Public Engagement Team for this blog. Ellie has been a brilliant part of the collaboration from the word go & was the clear-headed wisdom in the room on the last day we met & helped us look back at what we had achieved.
---
In all sorts of projects across arts, culture and engagement, we are often asked “what is the
impact?” or “how will you evaluate the outcomes of the project?”. Evaluation is sometimes
seen as time-consuming or onerous – but dedicating some time to it is rewarding and will be
useful to your project. You get to tell people (your supervisors, funders, colleagues…) exactly
what happened in your project and what all the benefits were of your work. Learning from
your evaluation will help you shape future projects, as well as deciding where your project
might go next.
Our collaboration was funded by an “Impact Acceleration Account” so we knew from the
start we’d need to tell the funders what we expected to happen, who we expected to
benefit and how we were going to measure it. From previous projects, we had also seen
how important the process of collaboration is as well as the final “product” (a workshop,
performance, resource…). We wanted to make sure we could evaluate both. Planning
evaluation from the start of your project helps to make it as useful as possible and measure
the things you care about finding out. It also avoids the rushed questionnaire created just
after the event has happened. Some useful questions to ask yourself at the start are: What
are you trying to find out? Why do you need to know? Who is the evaluation for? How much
time and resource do I have to spend on evaluation?
Next we thought about what types of impact we hoped to have on our audiences. We find
the Arts Council’s Generic Learning Outcomes a really useful way to think about all the
different ways someone might learn or benefit from an activity. Learning doesn’t just
involve gaining knowledge on something, you might change your attitude, gain a new skill or
change your behaviour – these all count too.
To evaluate this project we’ve used a “mixed-methods approach” where different outcomes
and audiences are evaluated in different ways. Methods included opinion wheels where you
stick a sticker on a poster based on statements you agree with, online and live evaluation
using sli.do and short interviews recorded on camera. This helped us to get a mix of types of
data and different levels of insight depending on what we needed. It’s important to think
about an appropriate method for the activity and audience as well as working out if it could
be threaded through the activity. We try to avoid taking people out of the experience or
making it feel like a whole extra task. People don’t usually want to fill in a 5-page long
questionnaire after an immersive performance workshop!
To evaluate the process of collaboration, there were several meetings and check ins
throughout at key milestones. This kept the project on track even as plans changed or
adapted as we went along. This reflective process is evaluative too and it is worth taking
notes or recording meetings for future reference. We finished with a focus group attended
by University staff and researchers alongside Kilter.
The focus group included tasks such as: recording ideas on post it notes, small group
discussions and individually drawing the “river” of the project and marking key milestones
where there was a shift in understanding, ideas or an obstacle. Part of the reason for
evaluating the process of collaboration is it helps inform what you might do next as
collaborators. We found that having dedicated time and space to talk as a group helped us
work out the strengths and challenges of the collaboration and gave us some exciting
potential opportunities to explore in the future.
We’re really looking forward to seeing where the collaborations go next and benefitting
from the evaluation data we collected along the way. We also hope to share the impact of
the project and build on this in the future. If you’re interested in finding out more about
evaluation and impact, check out the Public Engagement team’s Guides and Resources
page.
---
In all sorts of projects across arts, culture and engagement, we are often asked “what is the
impact?” or “how will you evaluate the outcomes of the project?”. Evaluation is sometimes
seen as time-consuming or onerous – but dedicating some time to it is rewarding and will be
useful to your project. You get to tell people (your supervisors, funders, colleagues…) exactly
what happened in your project and what all the benefits were of your work. Learning from
your evaluation will help you shape future projects, as well as deciding where your project
might go next.
Our collaboration was funded by an “Impact Acceleration Account” so we knew from the
start we’d need to tell the funders what we expected to happen, who we expected to
benefit and how we were going to measure it. From previous projects, we had also seen
how important the process of collaboration is as well as the final “product” (a workshop,
performance, resource…). We wanted to make sure we could evaluate both. Planning
evaluation from the start of your project helps to make it as useful as possible and measure
the things you care about finding out. It also avoids the rushed questionnaire created just
after the event has happened. Some useful questions to ask yourself at the start are: What
are you trying to find out? Why do you need to know? Who is the evaluation for? How much
time and resource do I have to spend on evaluation?
Next we thought about what types of impact we hoped to have on our audiences. We find
the Arts Council’s Generic Learning Outcomes a really useful way to think about all the
different ways someone might learn or benefit from an activity. Learning doesn’t just
involve gaining knowledge on something, you might change your attitude, gain a new skill or
change your behaviour – these all count too.
To evaluate this project we’ve used a “mixed-methods approach” where different outcomes
and audiences are evaluated in different ways. Methods included opinion wheels where you
stick a sticker on a poster based on statements you agree with, online and live evaluation
using sli.do and short interviews recorded on camera. This helped us to get a mix of types of
data and different levels of insight depending on what we needed. It’s important to think
about an appropriate method for the activity and audience as well as working out if it could
be threaded through the activity. We try to avoid taking people out of the experience or
making it feel like a whole extra task. People don’t usually want to fill in a 5-page long
questionnaire after an immersive performance workshop!
To evaluate the process of collaboration, there were several meetings and check ins
throughout at key milestones. This kept the project on track even as plans changed or
adapted as we went along. This reflective process is evaluative too and it is worth taking
notes or recording meetings for future reference. We finished with a focus group attended
by University staff and researchers alongside Kilter.
The focus group included tasks such as: recording ideas on post it notes, small group
discussions and individually drawing the “river” of the project and marking key milestones
where there was a shift in understanding, ideas or an obstacle. Part of the reason for
evaluating the process of collaboration is it helps inform what you might do next as
collaborators. We found that having dedicated time and space to talk as a group helped us
work out the strengths and challenges of the collaboration and gave us some exciting
potential opportunities to explore in the future.
We’re really looking forward to seeing where the collaborations go next and benefitting
from the evaluation data we collected along the way. We also hope to share the impact of
the project and build on this in the future. If you’re interested in finding out more about
evaluation and impact, check out the Public Engagement team’s Guides and Resources
page.
Comments
Post a Comment