Proving it: how music services can evaluate and evidence the outcomes of music making for vulnerable young people – part 2
See the first part of this blog here.
How could we capture this evidence: the music tutor’s perspective
A music tutor’s log is a simple but powerful way to capture evidence – and encourage reflective practice, critical in working with vulnerable young people.
It doesn’t need to be detailed – just a few notes at the end of each session that capture the tutor’s reflections and the young person’s progress in social, personal and musical outcomes.
We’ll be publishing a template tutor log soon – follow us on social media, or sign up to our enews to be the first to know when we publish new guidance and resources.
If you’re a music service lead or project manager, you might also want to ask your tutor to write a case study of the young person/young people, the progress they made and challenges they faced along the way. See this blog about sharing evidence [insert link to the Sharing evidence blog, see end of this blog].
Case studies can also become part of your staff training because they involve critical reflection by the tutor. We find that bringing tutors together in pairs to write case studies can be very beneficial too. It gets over the ‘blank page’ problem and often a tutor working alone will missed out or fail to spot evidence which seems ‘too obvious’. Although the case study is about the young person, it can help the tutor to think about what adaptation they made, what approaches they took that really inspired that young person to make progress. The reflective practice blog and this reflective practice video look at the benefits of reflecting in groups.
The young person’s perspective
This is probably best (ie most authentically) captured through conversation between the young person and the tutor as part of the session, and documented in the tutor’s log or a document co-created with the young person. A tutor can have an important role in helping a young person/young people to reflect on and express what’s happening for them, how it’s affecting them, and acknowledge when they’ve taken a step forward and what they need to focus on next (musically, personally or socially). They can also capture direct quotes which are particularly powerful.
If you’re a music service head or a project manager, it’s important that you’ve discussed with the music tutor what indicators of progress to record and what outcomes the programme is hoping to achieve. They’ll also want to capture relevant unintended outcomes/indicators.
The teacher/youth worker/parent perspective
Capturing evidence from a third party who knows the young person well helps to ensure a rounded and unbiased perspective (called ‘triangulated evidence’).
You could use a simple form or online survey; or capture video or audio feedback, perhaps at a performance or sharing of work.
The witnessing of young people in their ‘music’ persona can be really powerful, for the young people, who are validated, for the music tutor, who is recognised and for the teachers, youth workers or parents/carers, who understand, without you having to explain, what music can do for the young people they work with or care about. They may be surprised at the skill, team-work and increased confidence they show, and may well see a different side to them. Often these moments allow children in a bad place a moment of reimagining (and becoming) themselves.
Expecting the unexpected
Outcomes measurement should never become a series of hoops to jump through. It should be flexible, nuanced and able to encompass all types and styles of development, as well as unexpected results. Here are some examples:
· Leadership – the young person who draws the group together, resolves differences or quietly puts in extra hours on production may be exhibiting leadership skills as much as the person who’s opted to be the lead guitarist, solo singer, or conductor.
· Prior experience – one of our partners was surprised that a popular programme for young people with mental health problems, failed to show any improvement in emotional intelligence. In hindsight it is likely that the young people already had above-average emotional intelligence as a result of having undergone therapy, so there appeared to be little movement. Anecdotal evidence showed that there was lots of development in confidence, sociability and musical invention!
· Attendance – young people who attend irregularly will show less change, but these may be the very children who stand to benefit most from the activity. Don’t give up, use your outcome evidence to ask questions about what is really going on, for children, for settings and for tutors.· Time period – sustained engagement is likely to have a greater impact than a one-day workshop but both options will offer something to measure. If the outcome is social connection (an important aspect of wellbeing), an appropriate indicator for a 10-week programme might be ‘student develops mutually supportive friendships through the music activity.’ But for a one-day workshop this might be, ‘student reports feeling less isolated’.
Stay open to all the possibilities and don’t look at ‘no change’ or even ‘negative change’ as failure. Development is rarely linear, young people may appear to make progress and then come to a stop or even go backwards. Some present as very confident as a way of hiding their more negative feelings about themselves. It may be that you are measuring the wrong thing or that the participants don’t understand the evaluation questions. Use your outcome evidence to reflect and change: Do, review, improve.
Creating your tutor’s log as part of evaluation, click here
Sharing evidence with current and potential partners and funders, click here
[COMING SOON] Reflective practice log template