AUC Academic Conference 'From Virtual to Reality' The University of
Queensland 1996AUC Academic Conference 'From Virtual to Reality' The University of
Queensland 1996
Back to Home Page ...... by Presenter ...... by Paper Title ...... by Keyword
Paper Title:
The WebASSESS system:
online creation and management of testing
using the World-Wide-WebPresenter:
Dr Arshad Omari, Edith Cowan University
Authors:
Dr Arshad Omari, Lecturer
Dr Ron Oliver, Senior Lecturer
Edith Cowan University, Western Australia
Keywords: WWW, Student Assessment
Faculty area: Mathematics, Information Technology
Paper:
The use of the World-Wide-Web(WWW) for teaching and learning purposes is now commonplace for academic staff in universities throughout the world. Course content, supplementary materials and even complete courses are now available for study remotely using the Internet as a delivery medium. The visual nature of the WWW and the simplified nature of interaction with the information presented means that it provides the ideal means by which teaching and learning resources may be delivered for both on-campus and distance education/correspondence students.
The skills required of academic staff to author courseware resources for the WWW are easily learnt and are well supplemented by an ever increasing array of HTML editors available both free and commercially. While the creation and dissemination of resource-based teaching and learning materials is relatively straight-forward, the creation of more advanced interactive possibilities requires a greater knowledge of the WWW than simply HTML authoring. One such possibility relates to the use of self-assessment via WWW facilitiated online testing.
Student-self assessment is an increasingly important aspect of tertiary studies, especially as universities move into the age of on-line distance education. Managed multiple-choice and short answer testing, on-line, is one of many means of assessing student understanding of concepts and providing feedback. An important issue in the use of WWW for education is that academics who wish to use the system to its fullest require an in-depth appreciation of HTML, Hyper Text Transport Protocol(HTTP), the Common Gateway Interface(CGI), JAVA and/or Shockwave [1] (and copious amounts of free time).
A system is in development, WebASSESS, which will allow the generation of WWW-based on-line interactive testing materials without the need to understand HTML, CGI etc. In the first instance we are developing a multiple-choice question and answer environment in which questions are created and maintained by using a WWW CGI application interfaced via a browser such as Netscape. Students taking the on-line tests access questions and receive feedback via a WWW browser also. The WWW server is Macintosh-based and uses the WebSTAR HyperText Transport Protocol Daemon(HTTPD).
This paper describes the WebASSESS system at its current state of development. We will also address the issues which need to be considered when carrying out online testing; security, managing student testing and results, randomisation of questions from a pool, and the types of testing paradigms which may be feasibly offered.
The Department of Library and Information Science [2] at Edith Cowan University is heavily involved in the creation of courseware content for delivery via electronic means, both in CDROM and World-Wide-Web modes. We have created a number of courseware modules which cover a variety of content areas and make use of HTML, JAVAScript and Shockwave in order to provide some level of interactivity.
Although we use Email to facilitate interaction between staff and students using these materials, we needed a system which will support greater levels of interaction, especially with regard to student self-assessment. While most [70%] of our staff are literate with WWW authoring via HTML a system was required which allowed them to create testing modules and then embed these with their course materials.
The WebASSESS system is aimed at providing an online test management system used by academics/teachers to define and maintain pools of questions. In addition the system provides the means by which 'tests', based upon the defined pools of questions, may be defined and administered automatically by the system. The requirements we identified as necessary for an on-line testing scheme, to support student assessment and monitoring, include:
- multiple testing paradigms (eg. multiple choice, short answer) which may be automat(g)ically handled by the system;
- multiple pools of questions across different units/subjects/topics within which questions of different types may be defined;
- allow multiple users(academic staff) to securely maintain pools of questions, answers and tests;
- a simplified interface, preferably WWW-based, which allows the creation and maintenance of pools and the questions and answers contained within;
- allow the definition of 'tests' which can be taken using the system. Tests may be randomly generated from one or more pools, may be based upon particular questions which are pre-defined, or be a mixture of both;
- allow for online submission, checking and feedback;
- provide facilities for monitoring of student attempts at tests, allowing students to re-take a test but also providing the means by which this may be recorded;
- to allow the tests defined using WebASSESS to be attached to any WWW page, not simply confined to a be part of an overall testing WWW area.
Figure 1, below, shows the structure of the system we are implementing. The model is very much determined by its nature as on-line WWW-based testing scheme and relies heavily upon the modes of interaction possible via the WWW. It was decided to implement the system based upon the concept of the Common Gateway Interface(CGI) which allows external databases and applications to be interfaced to the WWW. Using WebSTAR, this mechanism is based upon appleevents(see Implementation below).
Figure 1 describes the major components of the system.
1. Two CGI-based programs; a test creation and maintenance module, used by academic staff, and a testing module used by students to take the tests defined. Hence there are two views on the system, the student and the academic.
2. A structured database which is organised into questions at the most basic level, pools of questions related in a manner which is defined by the maintainer of the pool (ie. topic or unit based) and the test which is defined to access questions from one or more pools. The CGI modules described above have access to different aspects of the database. Whilst the maintenance view provides question, pool and test definition functions, the testing view has direct access only to tests.
3. An authentication and security module which provides access and determines the authenticity of users and test maintainers.
A prototype implementation of WebASSESS is currently being developed using WebSTAR 1.3.1 for the WWW HTTPD, Applescript as the CGI engine and FileMaker Pro 3.0 as the database. The user interface views (academic and student) are provided via an HTML forms-based interface. The CGI applications reside on the server and are defined as compiled Applescripts which communicate with a single FileMaker Pro 3.0 database via appleevents.
The decision to implement the system using the above tools was made to ensure a prototype could be developed quickly, while at the same time allowing other technologies, in particular JAVA(see below), to become widely available on the Mac platform and hence generally useable.
Issues
The main advantage of using the WWW to deliver classroom materials is that it offers a simple, accessible, platform-independent interface. Whether students are on-campus or at some remote location, the student accesses the content and activities in the same way. This, of course, opens up its own problems including how to get content onto the WWW, how to authenticate and secure interactions and how to create good 'interfaces' which are operable across the numerous (allegedly) standard WWW browser interfaces.
In the planing and early implementation of WebASSESS, a number of issues became important:
Performance
The mixture of Applescript for CGI and FileMaker as the database engine means that the performance of the system, as currently implemented, is slow to moderate. Whether this is a dramatic issue or not is yet to be seen when we test the system with live subjects. Options to increase performance include implementing a native database module/CGI using C/C++ and making use of client side-processing using a mechanism such as JAVA applets(see below) rather than the forms-based HTML interface.
Security and Authenticity
Two essential security concerns are associated with on-line tests; the security of the online database of questions and the authenticity of the users of the tests(ie. avoiding cheating).
The issue of database integrity is handled in a number of ways by the system. Firstly, the Mac WWW interface(WebSTAR) does not provide access to any files outside the document hierarchy of the server. Secondly the use of a FileMaker database makes access online difficult without the ability to launch appleevents. Finally, all users allowed to maintain pools of questions require password-authenticated access.
The authenticity issue is more difficult as cheating can occur in many ways, such as pretending to be someone else or exchanging Email during tests. Other authors have solved these in a number of ways, the most promising seems to be the use of one-time passwords [3] which are issued just prior to a test being attempted. At present this issue has not been resolved in WebASSESS.
JAVA
The present forms-based interface to WebASSESS is inefficient as it relies on the maintenance and testing CGIs(Applescripts) to generate a fill-out-forms on the fly (in HTML) and then transport them across the WWW for display and use within a browser. The potential now exists using JAVA to create 2 applets one for maintenance and one for testing which communicates with the FileMaker database directly via the HTTPD. Essentially moving processing from server to client and providing a more dynamic interface.
Types of questions
The WebASSESS system prototype only support multiple-choice questions. Further extensions will include the ability to handle short-answer style questions, based upon model answers and keyword matching. More complicated question paradigms(ie. essay format) will require some human interaction in order to facilitate checking responses. It is hoped that WebASSESS will handle the forwarding of assessments between students and teachers using the same WWW-style interface.
Conclusion
This paper has outlined the structure, early implementation and development issues of a system, WebASSESS, which is aimed at providing an online test management system used by academics/teachers to define and maintain pools of questions.The system also provides the means by which 'tests', based upon the defined pools of questions, may be defined and administered automatically by the system.
Back to Home Page ...... by Presenter ...... by Paper Title ...... by Keyword
References
[1] The Shockwave plugin provides the ability to create multimedia content in Macromedia Director and embed this within HTML.Back to text
URL: http://www.macromedia.com
[2] The infob@Rn is the Department of Library and Information Science WWW presence and provides course materials as well as Department information. Back to text
URL: http://liswww.fste.ac.cowan.edu.au
[3] Dyreson, C. (1996). An experiment in class management using the World-Wide-Web, in the proceedings of AUSWEB96: The Second Australian World Wide Web Conference, Gold Coast, Queensland, 7th-9th July 1996. Southern Cross University Press. Back to text
Dr Arshad Omari
Department of Library and Information Science
Edith Cowan University
2 Bradford Street
Mount Lawley
Western Australia 6050Phone: (09) 370 6459
Fax: (09) 370 2910Email: a.omari@cowan.edu.au
WWW: http://kandinsky.fste.ac.cowan.edu.au
Back to Home Page ...... by Presenter ...... by Paper Title ...... by Keyword
