Evaluation of student facing web-based services: Final integrated report (CORE Education)

Key Results

The findings below are presented under headings derived from the specific questions posed in the research (see page 5). Where appropriate a specific instance has been provided to illustrate or substantiate the general finding.

1. Overall Impacts
   • In each of these educational services benefits were most obvious for students, as intended, and were much less obvious for schools and teachers. Services were aimed to attract users with particular needs, for example Studyit provided help for students involved in NCEA assessments and all services generally succeeded in attracting the targeted groups.
   • These services are emergent. The number of students using the services remains small when compared with the size of the potential target group. However Studyit is also quite large when considered in terms of online environments where a sustainable learning community is being encouraged
   • Certain groups were more likely to use the services. These services appeared to be used largely by students who overall felt confident about their schoolwork and who are motivated to seek out and expand their personal knowledge. Self-efficacy is an important asset of these student users. Fundamental learning dispositions of active users of these sites are likely to include students motivated to self-regulate their thinking and behavior and a readiness to inquiry where students are actively planning, monitoring, and modifying their thinking and applying effort to the task of  learning.
2. Impact variability
   • Due to the individualised nature of learning in online environments it is difficult to identify specific practices or conditions that are likely to impact on all users. These services were catering for niche groups of learners rather than all learners. For example, the inclusion of te reo Māori and Pasifika resources in WickED were encouraging greater use by Māori and Pasifika students. It is important for services to provide a diversity of ways of accessing the information needed to cater for the different ways different students access their learning. For example in Studyit almost twice as many students accessed the onsite information about ‘Subjects’ than viewed the discussion groups or forums. While 74% of AnyQuestions users had used the online chat facility, 24% had used ‘Do It Yourself’ answers and a further 24% had used the Search FAQs function.(NNR, 2006a, p.18) Use of the onsite information assists also to relieve pressure on experts who are providing feedback.
   • The majority of student use of these sites, especially AnyQuestions and Studyit, is from home. These sites are offering help and support for student when they are learning on their own.
3. Comparative Advantages
   • The nature and extent of the comparative advantages of these services include, the extension of teaching and reference services through a form of extended physical presence into the home or private study area of the students, individualised and personalised interactions with experts and even peers, student anonymity, the control students could maintain over the time and pace of interactions with an expert, and the ‘New Zealand-ness’ or specific cultural and schooling orientation of the sites for New Zealand students.
4. Curriculum Alignment
   • All three services align well with the New Zealand curriculum essential learning areas. Server stats and the NNR reports confirm that most students using these services seek help with particular learning areas, and these include mathematics, science and English. All three services provide support in these essential learning when students are working independently although AnyQuestions does not, due to the nature of the service, offer strong help with mathematics.
   • Only AnyQuestions and in Studyit supported a teaching presence in the form of highly subject specialist teachers and trained librarians for AnyQuestions and expert subject teachers in Studyit. Observable effective teaching practices in both services included questioning, re-tailoring tasks, modelling, support for inquiry and help with the process of deepening thinking around student driven topics, Other effective practices especially in Studyit included evidence of interaction with peers, co-construction of ideas, multiple perspectives, and reflection on problems submitted for discussion.
   • A number of challenges and opportunities for alignment exist within these services. Although the extent of alignment is variable, these services are well aligned with curriculum and effective teaching processes and practices within specific contexts of operation. For example, Studyit supports mathematics, sciences and English for senior secondary students involved in high stakes assessment. Services are also intended as additional learning support for students that designed to complement the work of schools not replace their role. Challenges and opportunities exist where these services particularly AnyQuestions and WickED could be more effectively incorporated into school activity structures.
5. Alignment with School Practices
   • AnyQuestions and Studyit are aligning strongly with the practices of schools. Both services have clear purpose and process, which is strongly linked to meeting students short and medium term, targeted learning needs. Both provide services that are strongly linked to the support providers have traditionally offered learners. A role of trained reference librarians has always been to support the process of inquiry and in assisting students and researchers in the location of resources and information. Senior secondary teachers have traditionally supported students through the process of high stakes assessment, with programmes of revision of curricula to be examined. WickED appears less aligned to the practices of schools. The service differs in that it is not utilising a traditional provider to offer a traditional service to students but is attempting to alter the way learning is perceived by students and teachers through introducing new and engaging activities (interactives) for students and teachers. Challenges and opportunities exist for WickED in convincing both teachers and students of the educative value of interactives and the processes utilised in the service. Assuming that schools are the norm the WickED content and process needs to align better with the work of schools.
6. Supporting Organisational Objectives
   • These services support the organisational objectives of the various project partners in varying ways and to varying extents. There is a strong connection between the service provision and the objectives of the project partners evident in both AnyQuestions and Studyit. For example the aim of the AnyQuestions project is to develop an online reference service for all New Zealand school students where they are only one click away from a trained librarian or subject specialist teacher. The challenges faced in realising this aim are both technical and the need for increasing resourcing for service expansion. Studyit is part of a range of services provided as part of the introduction of the New Zealand National certificate of Educational Achievement and the service supports students in a variety of ways in this. The challenges faced in realising this aim include the need to expand services into a greater number of essential learning areas while still retaining the educative value of the service. Opportunities exist for both services to include the types of web2.0 services that will improve goals for providers and align with students’ expectations for how web tools can work for them.
7. Unanticipated issues
   • Unanticipated benefits arising for learners, included the immediacy and relevance of the assistance provided, and in Studyit the emergence of student voice and knowledge co-construction. Unanticipated benefits were not observed for teachers, schools, school libraries, largely because these services were not being incorporated into classrooms and schools to quite the extent they could be. Unanticipated benefits for project partners, included online operators and teachers developing a far better understanding of New Zealand curriculum and student needs from working on online environments and feeling that roles were rewarding and satisfying.
   • Unanticipated issues arising for stakeholders, included the wait times for services AnyQuestion and the need to extend the operating times and extend the software affordances to allow for more operators to work concurrently. Although the AnyQuestions service works hard at ensuring a safe online chat environment for students there is reticence evident from parents and teachers about the use of online chat. The value of WickED for students as a support service and for teachers as a resource for classroom use needs re-examination.
   • A summary of the implications of unanticipated benefits and issues for service providers and policy making were identified as;
     • Given the perceived importance of the internet as a learning tool by students and parents there is a need to provide educationally relevant and appropriate learning spaces for a student group with diverse abilities.
     • Need to expand the reach of the existing successful models.
     • Need to better publicise the value of the current successful models.
     • Need to revamp and realign WickED so the service filling a clearly defined educative need, curricula purpose, and target group.
8. Learning Preferences
   • For these services to align well with young people’s preferences for their learning, it is important that interfaces are designed to aesthetically suit the targeted age group and that the site is easy to navigate and locate what is needed, and to clearly understand what the important functions are, that are embedded within the service. Also important is the ability of the service to engage young people by meeting their immediate demands for learning which may include the need to be entertained, to socialise, locate information or find help with homework and study. There also needs to be a strong link between school and teacher driven demands and what is available online to help students meet those demands. The ways and extent to which each of these services achieves these preferences varies. Of increasing importance to each of these services are the ongoing developments in softwares that encourage content creation and sharing. These functionalities, known as web 2.0, can be aggregated in various ways to alter learning environments in unique ways that give greater control to the learner. Students are encountering some of these functionalities in commercial websites, which is changing and raising student expectations about what they ought to be able to see and do on these websites.
9. Student Access and Management of Learning
   • While there is considerable evidence that the internet in general is beginning to change some of the ways in which young people access and manage their learning, there is little direct evidence from the suite of reports reviewed, as to the ways and extent these particular sites are individually changing the learning behaviour of young people to any large extent. There is however for a select and small group of students who are motivated to self-regulate their thinking and behavior, evidence that somelearning behaviours have been altered.
10. Student Expectations of Learning and Schools
    • These services are still in the early stages of development. The numbers of students accessing the services remains limited. It is too early in the development of these services to state to what extent they are impacting on children and young peoples’ expectations of learning and schools.
11. Continued success and sustainability factors
    • Learner success with online services is more likely when;
      • Student trust in the quality of feedback from mentor or subject specialist teachers and librarians is assured. This is a very important feature of both Studyit and AnyQuestions. The information students obtain is well aligned with their studies at school. Maintenance of a high quality of service provision is essential.
      • Students can feel safe and where teachers and parents can feel it is safe to recommend the environment for young students. Student safety underpins all three learning environments and is an important factor in adults recommending the site to students.
      • Young people can depend on access, through the technology, to ask questions at times which suit them. Studyit provides a good example of flexible dependable access. AnyQuestions ideally needs to be able provide greater flexibility of access to students. There should be no breaks in communication that can disturb or interrupt access.
      • Elements that contribute to sustaining student patronage and ownership include, strong links between school and teacher driven demands, guided services that offer relevance, focus, timeliness, and swift response times, and the provision of online services that are attuned to the design, navigation, linguistic, and communication attractants and repellents for young people.
    • Requirements for these services to be successful for learners and sustainable in terms of management include, maintaining a strong connection between specific educative purpose, tasks, and the targeted age groups, and to operate within a highly relevant and contextualised external environment. AnyQuestions provides a good sustainable model exhibiting strong project leadership, effective communication channels with stakeholders, and buy in from the different regional libraries involved, transparent management practices and a supportive advisory group.
    • Elements that are required for these services to be successful for learners and sustainable in terms of pedagogical development includes the involvement of teachers or guides, evidence of challenge and sustained thinking, and social interconnectedness combined with clever instructional design incorporating web2.0 tools aggregated in ways that encourage greater control over, and ownership of learning by students.
    • Requirements for these services to be successful for learners and sustainable in terms of service operation and resourcing include, the need for enough personnel and technical resourcing to ensure ease of access to operators and expert teachers at times that are flexible to student users. Publicity is required to promote the value of these services to users and recommenders alike.
    • Requirements for these services to be successful for learners and sustainable in terms of alignment with the wider learning network of young people involve, the design of generative environments in which the creation and sharing of content is not only textbased but includes a range of multimedia and to be able to engage with and critique that content in multiple ways. The building of personal webspaces such as blogs, the collaborative building of content in forums, as can be observed in Studyit, or in collaborative wiki development, the sharing of internet page content through co-browsing as in AnyQuestions, RSS feeds that allow users to subscribe to their favourite websites are a range of tools that young people encounter on commercial sites that are raising awareness and expectations that they can exert some control over these environments. Immersive sites and multiplayer gaming sites that involve strategy and team play are also contributing to raising awareness and levels of sophistication around the type of resources that are available on WickED.