ProjectProposal-Antonis Mannaris
From CS160 User Interfaces Fa06
Contents |
Introduction
The idea is to build a system that would facilitate a faster, more efficient and easier way to create and take exams, quizzes and other academic activities. This will include exams or activities at the class level, but also at a much larger, national or even international level. The main advantage of the system will be the automatic grading and processing of results on exam questions which allow computer grading (multiple choice, true/false), saving teachers (or academic agencies) a lot of time, effort, and error-prone grading. Even non-autogradable exams can use the system to their advantage, as I will explain later on.
Problem Description and Target Users
A lot of academic instructors today, including K through 12 teachers or even college professors, use computerised exams and exercises. At a much greater scale, the TOEFL (Test Of English as a Foreign Language) exam is taken on computers all over the world. The obvious advantage of this is the ability to produce results on the spot as a variety of questions can be automatically graded by a computer. In order for teachers to use such software however, their students need to have computer access, which is not yet possible in many schools around the country. Also, standardized tests like SAT are still taken by hand and then scanned because of the vast number of students that take it. Our system will enable teachers in less wealthy schools to have the advantages of computerized exams. Since the exam will still be taken on paper, the students or the school does not need to purchase computers, but the much more cost efficient ANOTO Digital Pen. The same applies to SAT exams, since the less bulky (than a computer) equipment of a digital pen will enable the current number of takers to remain for the same number of rooms, supervisors and other resources. So the system will target mainly teachers while at the same time making it possible for large scale exams to be accomodated. On the other hand, a conscious effort needs to go into the paper interface for the students taking the exam. The aim is to make the transition from the current system have as few changes as possible, thus ensuring that there will be no confusion introduced.
Outline
This bulk of the system design will be targetted towards teachers and exam preparing agencies like SAT's and GRE's. A lot of design will be devoted to developing an acceptable user interface to enable teachers to prepare exams, study and output results, and when necessary hand grade part or all of the exam.
Here are the main features:
- 1) An exam preparing program will provide teachers with an easy way to create several types of questions which will be compiled onto a quiz or exam.
- 2) Once the exam is prepared, two separate documents will be printed out. The questions can be printed out on regular paper, and an answer sheet will be printed out on the ANOTO paper where students will take the exam using an ANOTO Digital Pen. Alternatively (this will be the teacher's choice), both questions and answers will be printed together. The pens will be associated with a student either once (at the time of the exam) or at the beginning of a course. In either case, the system will keep track of which student an answer set belongs to by identifying the unique pen the exam was taken with.
- 3) Once the exam is completed, the teacher will be able to view the results in a variety of formats. Results over a series of tests may be compiled by student, class or some other group which the teacher will define at the beginning of a course.
- 4) If non-autogradable questions were included, the teacher will be able to grade them on the same answer sheet. This will enable the system to automatically compile the already graded answers with the additional grades the teacher will submit.
High Level Design
- 1) The main interface will be the program where exams will be prepared. This needs to be simple enough for teachers with just basic technical to use yet it needs to provide a flexibility and features that will make it desirable.
Ideally, the system will enable a teacher to hold data for all the courses she is teaching. This will include a roster with all the students in each class as well as some basic information about each class. This way, once an exam is created, it may be assigned to a class. To create an exam, the teacher will create a series of questions. For each question she will specify the question type (multiple choice, short answer, essay etc.), and the number of points for the question. More advanced features may include the subject of the question, difficulty and even a time limit which will be imposed automatically. For multiple choice questions the teacher will also need to specify the correct answer(s), so that the system may automgrade it once the exam is taken.
- 2)The system will then compile the questions onto one or two documents. These need to be precicely stored in the system so that answers may be tracked to their correct placement. The documents also need to be clear enough for the students to take the exam with no confusion. Question numbering and spacing are essential to this, as well as clear instructions for each question type.
NOTE: At this point I would like to bring up a challenging problem which I already foresee with the system. The problem of erasing and resubmitting answers. While streaming input from the pens means we can easily identify the last submission, we still need to overcome the issue of a)The student needs to be clear on how to make a correction and b)In questions that take up space, what if the student decides she needs more than the space provided.
- 3)The third aspect of the design is the processing of the results. As mentioned above, teachers should have the ability to view results by exam, by class, by student and other groups of their choice. Statistics like average, standard deviation are easy to produce and output with basic computation. Of course all these will need to be stored in a database system for future reference and use by the system itself.
- 4) While the submission of answers will identify a student by the unique ANOTO Pen that the student has, the teacher will be able to add comments and grade questions by hand OR on the computer. Once this is done, the complete statistics may be computed and the student may receive feedback either via e-mail or hard copy.
Further Insentives
Any teacher/TA/grader will tell you that grading exams is a time consuming and often error-prone task. Having such a system will definitely save time and effort on behalf of the teacher or academic agencies. Teachers today use computers to create the vast majority of their exams and activities, so such a system will not incur any additional cost or demand training on their behalf. So without forcing their schools or students to buy expensive computers, they can enjoy the benefits of what other teachers are already enjoying with online exams. This will improve the efficiency of the teacher as well as the student. Imagine how much more useful results are if you have them the day after the exam as opposed to a week later! If the system is used in national standardised tests, the cost of shipping and scanning millions of answer sheets will be erased. Instead, the whole process will be automatic and much less risky than it is today. Another insentive is that the automatic processing of exams means that records are kept around for school administrators and education agencies to use, without the need to manually input results. This will reduce the "edge" that more wealthy schools have over poorer schools today, where computerized tests are the norm. At both the teacher and administrative level it makes teaching much more efficient and provides a valuable tool to study the progress of students.
Solution Sketch
At the highest level, there are three main steps to an "exam cycle".
- A teacher needs to prepare the exam. Once this is done, the exam format, answers and other data are saved on the database and the exam documents are prepared.
- The students take the exam at the designated date and time. The answers are taken in, processed and the grading is fully or partially done. In either case results are stored for each student. Once the teacher grades any remaing questions, the results are complete.
- The results may then be compiled into statistics at the teacher's request. Also at the teacher's request, documents with her comments may be produced or emails sent to the students.
