AUC Academic Conference 'From Virtual to Reality' The University of
Queensland 1996AUC Academic Conference 'From Virtual to Reality' The University of
Queensland 1996
Back to Home Page ...... by Presenter ...... by Paper Title ...... by Keyword
Paper Title:
Student teachers learning to
assess school mathematicsPresenter:
Dr Tony Herrington, Edith Cowan University
Authors:
Tony Herrington, Jan Herrington, Len Sparrow and Ron Oliver
Edith Cowan University
Keywords: Assessment, Mathematics
Faculty area: Education
Students enrolled in teacher education courses are faced with learning both discipline content and methods of teaching content to schoolchildren. Within teaching methodology, a number of important pedagogical issues arise. One of the major issues is assessment. Student teachers are now required to learn about a variety of assessment strategies and be able to apply them on their school practices and later in their own classroom. Many of these assessment strategies are new to students whose main experiences have been assessment by unseen paper-and-pencil testing. Unfortunately, student teachers and teachers in general often employ methods derived solely from their own limited experiences as students (Ball, 1994; Lampert & Ball, 1990). Hence, despite the variety of innovative and effective assessment techniques, teachers generally continue to limit their means of assessment to a narrow range of pencil-and-paper methods (NCTM, 1995; AEC, 1991).
It would appear that teaching practice is a useful way of enabling student teachers to gain relevant experiences in which they can apply their knowledge gained from teacher-training courses. Unfortunately, not all students are fortunate enough to be placed in a teaching environment where they can explore and experiment with a variety of assessment techniques. Mousley & Sullivan (1995) have noted that preservice teachers' experiences in classrooms during their practicum have proved inadequate because often students observe teaching 'driven by texts and tests', or are ill equipped to detect the subtle differences between quality and mediocre teaching.
An environment that enables students to experience a range of appropriate classroom situations can be provided by multimedia. Recent advances in computer technology allow for the storage of large amounts of data in the form of visual, audio and text formats. These formats allow for multiple perspectives and representations of ideas to be easily accessible and interactive.
The approach that we have taken is to use current theories of learning, in particular the notion of situated cognition or situated learning, (Brown, Collins, and Duguid, 1989) as a framework for designing interactive multimedia that allows student teachers to become not only aware of different assessment strategies in mathematics education, but also to gain the conditional knowledge of when it is appropriate to apply them in the real context of the classroom. McLellan (1994) summarises the key components of the situated learning model as: apprenticeship, collaboration, reflection, coaching, multiple practice, and articulation of learning skills (p. 7). Many of the authors and theorists who are constantly refining the model believe that useable knowledge is best gained in learning environments which feature the following characteristics (Herrington & Oliver, 1995):
Design of the interactive multimedia package
Several considerations needed to be borne in mind as the program was designed. Preservice teachers using the program to investigate assessment strategies needed to be able to observe experienced teachers in the field demonstrating a range of strategies and techniques, and to then reflect on the most appropriate strategy to use in a particular situation. Assessment strategies were identified from the literature and discussions with colleagues. The strategies included on the disc are:
These strategies are grouped within broader categories of assessment such as teacher observations, teacher questioning, testing, interviewing, student reporting and student self assessment.
The context presented in the multimedia program needed to be situated in a real or simulated classroom, and authentic activities could require students to address the problems of assessment and to select their own alternatives to paper and pencil tests. The elements of the model to be provided by the learner, such as articulation and collaboration, could also be adequately catered for in the use of the interactive multimedia in the classroom. As such, the situated learning model, seemed to have a great deal of promise as an instructional design model to address the problems exhibited in the area of assessment in mathematics education.
Drawing upon the characteristics of the model, and the requirements of the content area of assessment, the learning environment was designed to comprise the following elements:
1. A CD-ROM on the issue of assessment in mathematics education incorporating:
- Video clips of teachers using various assessment techniques within their classrooms, with original sound;
- Video clips of teachers reflecting and discussing the strengths and weaknesses of the approach;
- Video clips of children discussing their feelings and thoughts;
- Interviews with experts in the field providing theoretical perspectives;
- Reflections by student teachers on the assessment approach;
- Text descriptions of each assessment categories;
- Teacher and student work samples;
- A problem-based notebook providing a variety of tasks within which to examine the resource.
2. An instruction book for facilitators on how to use and implement the resource, which would also provide advice on the situated learning elements which were not included in the program itself (such as collaboration and articulation).
Once the parameters of the content area had been sketched out, the elements of the multimedia program had to be determined.
Production of program elements
Video scenes
In order to produce the short video scenarios of classroom scenes, and teachers' and students' interviews, the assessment techniques were first classified into types, and possible scenes were suggested as a starting point for discussion with the teachers who were to teach the segments.
A video producer was consulted to assist with the filming of video segments, together with a sound recordist. They were required to film up to 30 sets of scenarios consisting of video clips of the classroom sequence, and interviews with the teacher and the students. Each strategy usually had a set of three video clips: the scene in the classroom, the student's comment and the teacher's comment. These were assembled onto a master tape ready to be digitised for the CD-ROM.
Student and teacher work samples
The students' and teachers' work samples were collected at the time of filming the video classroom scenes. If the teacher had set a written task for students, and these featured in the video, the work was collected and later scanned for inclusion in the program. When appropriate, the teacher's work has been given as the resource. All the scanned resources in this section of the program allow pre-service teachers to examine the actual documents which were used, or shown briefly, in the video scenes, and as such allow a more thorough scrutiny of relevant material.
Written descriptions of assessment strategies
The written descriptions of the assessment strategies were short 200-300 word essays describing each strategy and its use in mathematics assessment. These essays were written by the content experts and were based on current literature and international guidelines on assessment, such as those produced by the National Council of Teachers of Mathematics (NCTM, 1995), the Cockcroft Report (DES, 1982) and the Mathematics Curriculum and Teaching Project (Clarke, 1988). The essays were saved as document files and imported into the program.
Student-teacher advice
The advice from student-teachers was obtained from pre-service teachers in the last year of their undergraduate course, studying units taught by the two content experts. These students had completed a 10 week term of professional practice in schools. They were asked to give advice on using a particular strategy from their own experience on professional practice in schools.
Expert comments
The expert comments were acquired by a variety of means. The content experts tape-recorded interviews with a number of eminent mathematics educators, and further expert comments were requested by email from international contacts. Other comments were obtained by interviewing professors and associate professors from various institutions with known interest and expertise in particular areas. All the interviews were transcribed. Comments were edited for consistency and grammatical errors. The recorded interviews were made less colloquial in their transcription, and some were returned to the experts for final checking.
Electronic notebook
The notebook was designed to have two essentially different functions. Firstly, it was important to provide a work space for students to use as the central organising zone for their thoughts and observations which was accessible from any point in the program, and to copy relevant sections from any of the documents as appropriate. The second purpose for the notebook was to provide the problems and investigations that students might complete as they use the program. For example, one tab on the notebook takes students to a series of authentic investigations which replicate the kind of task a teacher might be faced with in real life. The tasks are presented to the student realistically, such as in a memo or letter, rather than simply a list of possible activities, and they include genuine constraints such as deadlines and time allowances.
Compiling the elements
The production of all the elements that comprise the total package took place on a number of different fronts, and in a variety of media. When all the components of a strategy were finished they were passed to the computer programmer who assembled the master version of the program.
The interface was designed to allow easy access to any of the program elements in a non-linear fashion (see Figure 1)
Using the program in the classroom
The finished program can be used with groups of preservice teachers who intend to teach mathematics in K-12 classrooms. The instruction book for facilitators suggests ways that the resource can be used and advises on the role of the lecturer. Optimal use of the materials is best obtained when students use the resource under the following conditions:
Focus of investigation:
The resource is best examined in depth, from a number of different perspectivesLength of time:
Best used over a sustained period of 2-3 weeks rather than for a single sessionNumber of students:
Students working in pairs or small groups around each computer, rather than individuallyTeacher support:
Teacher present during use to provide 'scaffolding' and support, rather than as an independent study activitySetting the task:
Teacher demonstrates the resource by thinking-aloud as an investigation is modelled. Students then choose an investigation from those provided, or their own choice.Evaluation
The completed program on assessment was evaluated with the assistance of a class of 12 preservice primary teachers studying mathematics education method. Students were asked to work in small collaborative groups of 2-3 students. The activity given to the students required them to assume the identities of new teachers in a school given responsibility to prepare a report to staff on assessment strategies.
The entire class was observed using the resource. In addition, two students were videotaped and were interviewed after the class. The students were interviewed about the elements of situated learning which had been incorporated into the design of the materials, and whether they felt these elements contributed to their learning. They were also questioned about design aspects of the interactive multimedia program itself, such as the interface design.
Students responded positively to the interface. The students appeared to conceptualise the layout of the various resources and their contents very quickly. One student mentioned that you always knew where to find things. The one feature almost all the students disliked about the program was the size of the video picture (a trade off between the limited memory capacity of CD-ROMs and the number of video clips). All the video clips appeared in the area given to the television screen, a space with dimensions of approximately 5cm by 4cm. However, one student mentioned that the size of the video was compensated by the ability to scrutinise any illegible documents shown on the video from another source, in this case, the samples drawer.
The navigation system in the assessment program was designed to provide referential linking to enable students to access any media element or document, together with the notebook and the help screen. Generally, students had very little trouble acquainting themselves with the navigational systems provided by the program, and they readily accomplished the means to investigate the resources. Students also liked the non-linear layout of the program. The freedom to access material in the order of their own choosing was commented on by a number of students. For example, this student noted that the program did not force the user to complete elements of the program in sequence:
You could go through and do whatever you like. It doesn't say you have to do this bit first, and then that bit. You could go and do whatever. (Interview with Deborah)
Search strategies employed by students varied considerably between groups. Students could choose to approach the search systematically, or use an unstructured path through the program. Students who approached the task systematically generally opted to investigate the resource by strategy (the assessment strategies written on the whiteboard in the interface) or by media element (the video clips or the documents in the filing cabinet).
While the interface and navigational paths of the program were deliberately kept as simple as possible, the conceptual content was far from simple (the package included over 60 video clips and over 80 documents). This was acknowledged by one student:
I was actually surprised at how much you can learn from such a simple brief program. There was just so much in there that you could learn from. (Interview with Deborah)
The predominant feature of the context of the assessment program was that students valued the real-life relevance of the material they were using. They frequently pointed out the contrast between the authentic context presented in the program and the decontextualised approach frequently employed in their courses:
Instead of just showing us the theory, it also showed the scenario inside the classroom, so we can do that when we go on prac. It gave practical examples which I think the course is lacking a lot of. (Interview with Deborah)
Other comments relating to the context were that students could relate to the episodes and saw them as being very lifelike. One student related the feeling that she could go beyond the computer representation of the program into the classroom itself.
The authentic activity was designed to incorporate all the uncertainty and unpredictability of an authentic task and to allow students to apply sustained thinking on a single topic over a lengthy period of time. The program also presented a variety of perspectives on each assessment strategy, from the teacher's, student's, pre-service teacher's and expert's point of view. Students were very positive about the variety of sources of information presented on the same strategy. One student, comparing the program to a traditional lecture, indicated that the multiple perspectives provided many 'avenues to understanding':
In a lecture you can't click onto the video and get the video to play. When you've got a huge lecture situation, the lecturers can't keep stopping and going ... Whereas with this it gives you so many avenues to understand it from ... There's just so many different ways of looking at that one strategy. (Interview with Deborah)
Students perceived many clear advantages in working collaboratively. One recurring advantage, was the benefits of articulating their knowledge to their partners. The following comment was typical:
When we were doing our notes ... you'd have to explain and explaining always clarifies no matter what you're doing. (Interview with Glen)
Reflection is a very personal aspect of learning, and as such, no program can force students to reflect as they use it. However, in order to provide a learning environment which would promote reflection, the assessment program was designed to enable multiple entry point, non-linear navigation, and access to the electronic notebook. Students using the program frequently mentioned the electronic notebook as an aid to reflecting on their learning.
You could put a little note in the notebook and then you could go back and look at the next bit. The notebook helps you ... if it hadn't been there we would have just read it all through and then said 'What did it all mean?' So the notebook really helps you reflect. (Interview with Glen)
Students articulated their understanding of assessment strategies in two ways: the formal report to the staff meeting, and in their discussion with their partner as they used the program. Many interactive multimedia programs do not allow either form of articulation to occur. Students are assigned individually to computers to work alone, and in so doing, the knowledge remains tacit. Students using the assessment package were very much aware of the value of formally articulating their learning in presentation of reports to their classmates:
Because we had to give the report meant that we looked at the bits we were doing more carefully, in that we had to know what we were going to say. (Interview with Glen)
Coaching and scaffolding is generally seen by the students as an important aspect of the learning process. The role can be undertaken by both teacher and student partners in collaborative learning situations. An important concern of students was that it was essential that assistance be available at the time of need, and that failure to attend to these immediate needs would result in time being wasted.
From this preliminary research, the theoretical framework of situated learning used to design the assessment program appeared to be very successful as a model of instructional design for this type of software. Further, more comprehensive, systemic research on the use of the program is planned to be conducted in the near future.
Conclusion
Current assessment practices in mathematics education demand change. For change to occur teachers need to see and experience approaches in real classrooms with real students. Additionally, the ownership of these strategies is more firmly anchored when students can make their own judgements based on honest perspectives offered from a wide range of key players. The multimedia package described in this paper goes some way to meeting criteria necessary for helping students learn to assess mathematics.
Back to Home Page ...... by Presenter ...... by Paper Title ...... by Keyword
References
Australian Education Council (1991). A national statement on mathematics for Australian schools. Carlton, Vic: Author.
Ball, D.L. (1994, April). Mathematics and teaching through hypermedia. Paper presented at the 72nd Annual Meeting of the National Council of Teachers of Mathematics, Indianapolis.
Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42.
Clarke, D. (1988). Mathematics curriculum teaching program - Assessment alternatives in mathematics. Canberra: Curriculum Development Centre.
Department of Education and Science (1982) Mathematics counts: Report of the Committeee of Inquiry into the Teaching of Mathematics in Schools under the Chairmanship of Dr W H Cockcroft. HMSO: London.
Herrington, J. & Oliver, R. (1995). Critical characteristics of situated learning: Implications for the instructional design of multimedia. In J. Pearce & A. Ellis (Eds.). Learning with technology (pp. 253-262). Parkville, Vic: University of Melbourne.
Lampert, M., & Ball, D.L. (1990). Using hypermedia technology to support a new pedagogy of teacher education (Issue paper 90-5). National Center for Research on Teacher Education.
McLellan, H. (1994). Situated learning: Continuing the conversation. Educational Technology, 34(10), 7-8.
Mousley, J., & Sullivan, P. (1995). Learning about teaching. Geelong: Deakin University; ACU.
National Council of Teachers of Mathematics (1995). Assessment standards for school mathematics. Reston, VA: Author.
Dr Anthony John Herrington
Chair, Department of Mathematics Education
Edith Cowan University
2 Bradford St
Mt Lawley
Western Australia 6050Phone 619 370 6474
Fax 619 370 2910E-mail: a.herrington@cowan.edu.au
Back to Home Page ...... by Presenter ...... by Paper Title ...... by Keyword
