Travellers is an early intervention programme run by Skylight for students (generally in Year 9) in New Zealand secondary schools.In order to build on existing studies on Travellers, Skylight commissioned the New Zealand Council for Educational Research (NZCER) to conduct an external evaluation that explored the short- and medium-term outcomes for the young people who took part in this programme in 2008 or 2009. This study began in April 2011 and finished in January 2012. The overarching evaluation questions were:
- What are the short- and medium-term impacts of the Travellers programme for young people?
- How well is the Travellers programme meeting the needs of particular target groups?
- How can the Travellers programme be improved?
Methodology
To collect information about the impact of Travellers on the short- and medium-term outcomes of Travellers, we conducted a survey focusing on the opinions of young people who had completed the programme a few years ago.
The short-term outcomes that were the focus of the evaluation were:
- access to appropriate youth-focused sharing and learning opportunities
- improved connectedness to school
- trusting relationships formed in the group
- improved help-seeking skills
- increased access to appropriate support.
The medium-term outcomes that were the focus of the evaluation were:
- increased resiliency and ability to navigate changes and challenges
- engaged, confident and motivated young people
- more positive relationships with peers/families/teachers.
We also included questions that had been used in Skylight’s initial screening survey so that the responses from the baseline data could be compared to the follow-up responses to tell us about shifts over time in relation to the above outcomes.
In total we received 212 completed survey responses from students from 26 different schools. Reflecting the fact that Travellers was first introduced in Auckland schools and has only been a national programme since 2008, many of these schools were from the Auckland region. This resulted in a higher proportion of Pasifika students completing the survey than is found in the general population.
Along with exploring outcomes for all of the young people in the evaluation, we also looked at the outcomes for students from different sub-groups, including groups that previous research had shown were at higher risk of presenting suicidal thoughts and behaviours (i.e., Māori students, Pasifika students and students from low socioeconomic communities).
We also validated the survey data findings through qualitative focus groups with students (pilot and post-survey). These three sources of data served to give us a more robust picture of impacts than any one source.
Key Results
Overall, Travellers appears to be a useful programme for a range of students, with the majority (82 percent) rating Travellers at the top or mid-point on a five-point scale. The majority also reported that Travellers had helped them to learn a range of strategies, in particular strategies that contribute to increased resiliency and ability to navigate changes and challenges, and positive relationships and help-seeking. For example, over two-thirds (69 percent) had asked for more support since doing Travellers. This support came most often from friends, followed by family, the guidance counsellor, and teachers or deans at the school.
Travellers was less likely to help with strategies relating to schoolwork and time management, although it is unclear whether this is because students already have these strategies or whether this is less of a focus for Travellers.
Over half of the young people considered Travellers was run at the right age for them (64 percent). Sixty-two percent said that they could trust the people in their group, and a further 22 percent answered neutrally.
The students selected to take part in Travellers mainly came from a group rated as being at high risk (as indicated by criteria on the screening survey). Since taking part in Travellers, at the time of follow-up the young people reported experiencing less subjective distress despite experiencing a greater number of challenging life events compared to when they completed the screening survey. This increase in experience of challenging life events is not surprising and is likely to be due to maturation. The fact that the young people had lower levels of subjective distress suggests that since taking part in Travellers the young people had improved their ability to manage challenging life events.
Overall, at the time of the follow-up the young people had fairly positive views of themselves and their lives, with the median score of 32 on the Subjective Experience of Distress Scale (with 40 or higher considered as at risk). There were also fewer young people who said they did not feel good about themselves most of the time since taking part in Travellers.
Some young people felt that the strategies they learnt in Travellers are very useful in dealing with challenging life events. However, less than half (44 percent) of the young people said that Travellers helped a lot or a bit with a difficult situation they were asked to describe in an openended section of the survey. It appears that while young people can identify skills and strategies they learnt in Travellers that contribute to their resiliency, it is more difficult for them to identify how the strategies they learnt in Travellers can be applied. It is not necessarily that these skills have not been applied to new situations; it may simply be that it is difficult to articulate this complex process.
However, the young people’s open-ended comments clearly indicated that Travellers had positively affected their general wellbeing. Many were clearly able to articulate their feelings and experiences and made connections between their current sense of improved wellbeing and Travellers. Given that the survey required the young people to recall experiences and views about themselves from two or more years ago, the fact that many were able to do this suggests that Travellers has had a positive impact.