Teaching, learning and assessment are interpersonal activities. When making an assessment on a student, it must be acknowledged that although a teacher’s decisions are based on professional judgment, sometimes there will be an emotional component also [O’Connor, 2000: 15]. How teachers feel about individual students or groups of students will affect their judgments. This research investigated how teachers’ perceptions of students, gained from working with them intensely over a week or from seeing them interact as they negotiated a story line, affected their assessments of the speaking performance of Year 8 students presenting a puppet play. It also looks at how moderation of the assessment process impacted on the consistency and reliability of teachers marking using set criteria. Unlike previous research it would seem that neither moderation nor discussion between markers achieved consistency across markers. Instead it would seem that teachers’ perceptions of such things as students’ leadership in developing the storyline had a much greater impact on how they allocated marks for students’ spoken performances.
When assessment for areas, such as speech performance, involves observation there will be bias, prejudice and value judgements made by the teacher. Observation involves a combination of senses which selectively filter information. What is seen, heard or felt during observation is influenced by our existing concepts, knowledge, ideas and expectations of who and what is being observed [Harlen, cited in Conner, 1991: 63]. It has been suggested by Holloway, Hardwick, Morris and Start [1967 cited in Rowntree, 1987: 85] that assessors involved with oral assessment may be influenced by what they see of a student’s personality rather than by the student’s abilities. Favoured personalities are given better grades than their performance appears to justify.
That assessment is not always an objective process and may contain subjective elements must be recognised [O’Connor, 2000: 14]. It is therefore important to investigate how teacher’s perceptions of students affect their ability to assess those students performing a task. It is also important to see if teachers are aware of the influences on their assessment grading. Research done by Hurman [1996] looked at subjectivity involved in marking GCSE role plays in foreign languages. Although looking at communication in a language other than English, Hurman investigated how markers made judgments of oral language performance against marking criteria. Hurman looked at the extent of the variation between marks awarded to see if more inter-marker consistency could be obtained. Hurman found that increased reflection time before allocating a mark caused less variation in grades allocated. However, he still found that inter-marker variation in communicative assessment is an inevitable part of the process and needs comprehensive measures to control it in the interests of fair results [Hurman, 1996: 21]. For Hurman, much depends on the intensity and scope of the moderation process and careful reviewing of deviant markers in order to alleviate the subjective element of assessing oral language.
Gillian Wigglesworth [1994] also looked at marker behaviour in the assessment of oral interaction in an ESL setting and found that while marker training may reduce variability in marking, it does not entirely eliminate it. Training was found to encourage markers to be more internally consistent in their use of marking schedules and their judgment. However, there were a number of factors found to affect oral language assessment results. These were:
- Some markers were consistently harsher/more lenient than others
- Some markers rated a particular sub group of the test population more harshly
- A particular assessment criteria was harshly/more leniently rated
- Different interpretations of rating scale
- Lack of self consistency in marking [McNamara, unpublished manuscript , cited in Wigglesworth, 1994: 77]
- Although her sample size was small and conclusions tentative, Wigglesworth found that when markers were evaluated in relation to how they did the marking, they were then able to incorporate the feedback into subsequent marking situations. A further question was raised as to whether change in marker behaviour could be maintained over an extended period of time but no research was carried out on this.
It is important that when observing and making judgements on student’s abilities assessors are able to explain and justify their interpretations. As well as being aware of how perceptions of the students’ affected their marking there was also a need for teachers to understand how their own previous experiences may affect their marking. What is being observed is affected by the state of the observer and their past experiences. It is important, therefore, in order to improve observations, to work with another observer, who will see things differently and thus enlarge and extend one’s seeing [Dean, 1983, cited in Conner, 1991:64].
The National Education Monitoring Project (NEMP) currently uses teachers to mark assessment tasks which require subjective decisions. It is believed that teachers have the experience from which professional judgements can be made. As part of the marking, moderation of several examples of performances are done before and in the middle of the process. It is anticipated that this moderation would support the teacher markers in being consistent in their use of the marking schedules. For the marking of team performances involving four students, often markers work in pairs. This involves them watching the videoed performance and then discussing the options before awarding a grade of each of the set criteria. This is also hoped to encourage more consistency in marking to the criteria. However, given that previous research by Hurman [1996] suggested that consistency is hard to achieve between markers, there was a need to see whether background information about the students contributed to changes in the grades awarded and if there were changes what type of background affected the grades.
Usual NEMP procedure would mean that only the performance of the play would be assessed. However, NEMP video recording included the negotiation and planning phase of this task. Rowntree [1987: 173-175] suggested that in order to provide a more accurate assessment of a student’s abilities it is necessary to consider what goes on in the process as well as the product of an activity. On the other side, as stated by Nambia and Goon [1993: 20], negotiation, as a form of oral interaction, is a complex process; it requires more paralinguistic and extra linguistic support.
The impact of having teacher markers who were also the teacher administrators, and therefore have prior knowledge of students involved in the tasks, had not been considered previously as a factor that may influence assessment grades. It was necessary to discover if the information relating to certain students that a teacher administrator brings to the marking process affected their judgement of student performance. We also wanted to know whether viewing each team’s practice assessment would result in markers altering the grades they give. In particular it was important to see if additional information about students obscures or enhances the teacher’s ability to judge a speaking performance against set criteria. Therefore our research questions were:
- Does background knowledge about a student affect a teacher marker’s perception of their performance in an assessment task?
- Does observing students in the planning stage of their performance affect teacher perception of their performance in an assessment task?
- If background information does affect teacher perception of student performance, in what ways are teacher perceptions altered?