Evaluating outcomes: How to use ‘Capturing Magic’

A Practical Guide to using NYCI’s ‘Capturing Magic’ – a tool for evaluating outcomes in youth arts projects in line with the National Quality Standards Framework

The demands on youth workers continue to grow, year on year. We are required to justify our work through outcomes and gathered evidence. Funders require a certain type of language, which at times feels like jargon and maybe unconnected to our everyday practice in youth arts. If you are like me and struggle at times to find the right words to name what you are doing, then the NYCI Capturing Magic- Youth Arts Evaluation Tool is the perfect resource for you.

The National Quality Standards Framework (NQSF) is designed to support the youth work sector, improve the work that practitioners do and help capture and demonstrate the impact it has. The five core principles of the framework establish that youth work should be:

1. Young person-centred: Recognising the rights of young people and holding as central their active and voluntary participation;
2. Committed to ensuring and promoting the safety and well-being of young people;
3. Educational and developmental;
4. Committed to ensuring and promoting equality and inclusiveness in all its dealings with young people and adults;
5. Dedicated to the provision of quality youth work and committed to continuous improvement.

I often get a feeling of being overwhelmed when it comes to programme development, funding applications and report writing, in addition to crucial everyday tasks (doing the youth arts work!) under the time constraints we work with. It is often then that I turn to ‘Capturing Magic’ to help me out.

The intention and elements of Capturing Magic align with the NQSF principles in many ways. Firstly, the outcome areas focused on are: creative ability; emotional and mental wellbeing; social confidence and team working; being included and participating in society; and life skills. These overlap well with the NQSF results areas, which similarly relate well to the ‘Better Outcomes, Brighter Futures’ framework: Ireland’s first overarching, whole of government, policy framework to set out how the country intends to do its best for children and young people (0-24 years).

The NQSF requires a scale of attainment, documentary evidence and your observation on groups. All of these requirements are addressed within the design and evaluation approaches, tools and templates outlined in Capturing Magic.

The evaluation tool provides a scales template that helps you record the progression and development of your youth arts programme. It also illustrates an outcomes tree, which is further explained throughout the resource. That is designed to capture a series of observable behaviours in young people, which in turn reflect underlying aspects of a young person’s attitudes, confidence and commitment that can arise from engaging with an art form and the creative process they experience. I currently have this outcome tree (p5 of Capturing Magic- http://www.youtharts.ie/sites/default/files/CapturingMagic%202017-acc.pdf ) attached to my desk. I find it helps the creative flow during the planning and evaluating stages.

There is an example of a write-up on (Capturing Magic, page 41 of online version), which is a good starting point to get inspiration for report writing.

I believe strongly in sharing resources and adding constantly to your toolbox as a youth arts worker. My step by step guide to planning, delivery and evaluation of a youth arts programme is as follows:

1. Identify the current needs of your group: for example, artistic & emotional confidence/ technical/ embracing difference/ resilience, etc. (Page 11 of online version Capturing Magic), which relates to NQSF guidelines regarding assessment of needs on pages 14-21.  http://www.cdysb.ie/Files/NQSF-Planning-2c-Needs-Assessment-and-Evaluation-M.aspx
2. Design programme with group and artists: see the Creating Magic resource (another great reference guide) for more details on ensuring strong participation. http://www.youtharts.ie/sites/youtharts.ie/files/CreatingMagic.pdf
3. Plan ahead for results: Use Capturing Magic to aid and strengthen programme development and implementation by using outcomes and scales appropriate to your group needs. I usually go with 3 or 4 main points/scales provided in Capturing Magic (pages 11-40 online version) that are most relevant to the group. This would address the NQSF principle of quality youth work provision and help you to document your key learning/ progression paths.
4. Gather evidence and document progression: Making a plan to identify intended results (using Capturing Magic’s outcomes and scales, for example) allows you to monitor progress, which addresses the NQSF principles of dedication to quality and continuous improvement of equality/inclusion.
5. Evaluate and Review: By following the previous steps, you have the basis for the evaluation and review you need to assess the impact of your programme after finishing it, which is needed to fulfil NQSF requirements around tracking educational/ development gains, dedication to quality and continuous improvement.
6. Start again.

NQSF requires plans, implementation and continuous improvement details, regular cyclical reports, and evaluations at key points. Capturing Magic supports youth arts workers in considering the anticipated outcomes from the start, as well as providing the language and tools needed to capture and describe outcomes that are often quite difficult to articulate.

By encouraging youth arts workers to map out priority outcomes and ways of measuring impact, Capturing Magic is both prompting and supporting us to gather the evidence and documentation the NQSF requires, in the type of language expected.

The clear, concise and aesthetic layout of Capturing Magic also means it is accessible and user-friendly. So don’t be afraid to pick it up: you will be very happy you did because it will make planning, progress mapping and reporting a whole lot easier for you.

Download printable version here