In 2005, the researcher conducted a probe study (Knight, 2005) in which the NEMP assessment tasks from 1996 to 2003 were considered to identify those which had significant potential for assessing different kinds of thinking. One of the objectives of the research was to identify particular tasks, presented in a one-to-one interview task approach with videotape, which might be used in subsequent studies to explore the nature of the thinking which was actually used by a sample of students in completing the tasks.
The classification of thinking used in the study was that of the New Zealand Curriculum framework document (Ministry of Education, 1993, p17) which states, among other attributes, tha
Students will:
- Think critically, creatively, reflectively and logically.
For the purpose of the research, the following working definitions of these kinds of thinking were:
- Critical thinking is thinking which involves evaluation and, perhaps, challenge.
- Creative thinking is directed towards solving a problem in one’s own way. It often involves imagination and initiative.
- Reflective thinking involves looking back on one’s previous thinking, knowledge and understanding.
- Logical thinking is directed towards making deductions or presenting arguments.
(Knight, 2005, p3)
The terms are used in the same way in this study.
In identifying suitable tasks for subsequent studies a test format suggested by Halpern (2003, p 361) was used. She writes:
- “My own preference for test format, when the goal is to assess critical thinking, is to use an ecologically valid example with an open-ended response format, followed by specific questions that probe the reasoning behind an answer.”
The NEMP tasks which have the potential required are those which:
- are in a one-to-one interview format
- are open-ended
- ask for explanations or justifications
The research identified 22 such tasks. 11 of these were critical thinking tasks, 10 reflective thinking tasks and one logical thinking task.
In this study the responses of students to some of the critical thinking tasks is examined. The 11 critical thinking tasks identified came from four different assessment areas: Science, Art, Technology, and Listening and Viewing.
The best critical thinking task from each of these areas was selected and the videotapes of random samples of about 30 year 4 students and 30 year 8 students responding to these tasks were analysed.
Purpose
Research questions:
- What is the nature of the thinking actually used by students in completing the critical thinking tasks?
- Are there differences between year 4 and year 8 students?
- Is it possible to identify differences between subject areas in this regard?
Intended outcome:
A better understanding of the nature of the thinking of students in completing the NEMP assessment tasks.
Use expected to be made of the findings:
The findings may be useful to:
- NEMP in designing further tasks to assess critical thinking skills.
- Teachers who wish to encourage and assess critical thinking skills in their classrooms.
Methodology
The choice of tasks for analysis:
The four tasks chosen for this study were selected on the basis of the work done in the 2005 Probe Study (Knight, 2005).
The tasks, and the reports from which they were taken, are:
Report 13 Science 1999 Page 53 Environmental Issues
Report 18 Aspects of Technology 2000 Page 22 Nut Cracker
Report 10 Listening and Viewing 1998 Page 26 Looking Around
Report 2 Art 1995 Page 43 Two Sculptures
The student sample:
The Educational Assessment Research Unit at the University of Otago were asked to supply 30 videotapes, selected at random, at both year 4 and year 8 for each of the four selected tasks, a total of 240 tapes.
At EARU 31 ID numbers for each task were selected expecting that there might be occasional tapes missing. There were also a few tapes without recordings of the task required or which were inaudible. In the end 231 tapes analysed. The numbers of tapes analysed for each task are given in the Report
Analysis:
The tasks
Each of the tasks was closely examined looking at:
- the marking criteria for the task
- the reporting of student performance
- the involvement of the interviewer in administrating the task
- the nature of any probes suggested
- the nature of the thinking which it seemed likely would be required to complete each part of the task.
The videotapes
Initially all the videotapes were viewed, focussing particularly on those parts of the task which required critical thinking. Brief notes were written for each student and those tapes containing responses which the researcher thought might be worthy of further consideration were identified.
Each subject area was then considered in turn. The tapes which had been identified as potentially useful were revisited and some of the responses were transcribed.
Comments were written concerning the responses in each subject area
The results of this consideration are presented in the next four sections of this report.