This report presents the findings of a small scale piece of qualitative research with nine providers delivering programmes to young people. The purpose of this research is to understand the experiences of implementing the Assessment Tool. This report supplements the quantitative analysis of first results from the Assessment Tool reported in Lane (2012). There is interest in both the results coming out of the Assessment Tool and the experiences of implementation.
Purpose
The research addressed two key questions:
- What was the experience of organisations and educators in implementing the Assessment Tool and using it to support teaching and learning?
- What needs to be taken into account from these experiences in analysing the data available from the Assessment Tool?
Methodology
The research was undertaken with providers offering fees-free places within the Youth Guarantee programme in 2011. An online survey was conducted with all 35 providers who offered these places. Nine providers were selected for follow up case studies. Focus groups were also conducted with three groups of learners from two large providers. The commentary on data analysis was added after the research was completed.
Key Results
The interviews confirmed that successful implementation is more than just setting up learners to complete the assessments. It needs to be part of wider organisational engagement with developing the literacy and numeracy skills of learners. It requires:
- Educators who understand the importance of literacy and numeracy and have the skills to be able to embed relevant tasks and activities into the vocational training they deliver
- Educators who also understand the use of assessment for learning and value the use of assessment information for diagnostic purposes and tracking whether their efforts are having an impact
- Learners who are engaged in the learning process, understand why the assessment matters and can make sense of the results for themselves.
- Organisations that have the resources and processes to support educators in using the Assessment Tool in a timely manner.
A consistent issue raised by all organisations was the challenge of engaging learners with the assessment process. This was a particular concern with the end-of-programme assessments. Learners have often had negative prior experiences of assessment. Younger learners found the questions did not always relate to their life experience. They needed to understand the purpose and benefit of the assessments for their own learning and development. The assessment is seen as something they do for the educator, it doesn’t contribute to their grades and they can find it difficult to fit into their own understanding of their skills and learning. If they are not convinced and engaged with the process then the results may under-represent their full potential.
The key message for data analysis from this research is that the Assessment Tool is administered within a ‘real world’ context. Organisations and educators have flexibility about which assessments to use with which groups of learners and when to assess. Assessments are undertaken in a range of different circumstances. The level of engagement of educators and learners with the Assessment Tool can be variable. All of these factors need to be considered in drawing conclusions from the data. The Assessment Tool results should be regarded as one source of information alongside others in understanding learner outcomes.