GroupBrainstorm-Group:BigBrother
From CS160 User Interfaces Fa06
We are totally watching you
Contents |
Brainstorming
- KINDS OF EXAMS
- long-distance testing
- multiple-choice exam
- in-class exam
- ACCUMULATING STATISTICS
- time spent on problems
- time spent between questions
- have digital records of submitted tests (help with regrades)
- have digital records of how the teachers graded the test
- group questions into topics and evaluate students’ performance for each group
- useful in identifying which area of studies to focus on
- faster categorization of questions to determine the amount of points to be awarded
- based on the amount of time the students spent on
- based on difficulties
- PROMOTING ANTI-CHEATING
- detect whether students write before/after time is called
- monitor what students have written or are writing while taking the test
- compare students’ exam behavior
- automatically red-flag students
- ASSISTING STUDENTS DURING/AFTER EXAM
- allow students to send messages to teacher/GSI
- confusing questions/requesting help
- reporting cheating
- asking for bathroom break
- allow people to write answers down in any order
- computer will put everything back in order
- help students evaluate their test-taking behavior
- provide digital feedback to students (containing the statistics above)
- GRADING EXAMS
- automatic grading
- determine if a student’s answers need to be considered
- recorded student’s behavior may convey his way of thinking, which can be considered for giving half credits to his wrong answer
- fast comparisons of exams taken late and everyone else’s
- can automatically/digitally mark answer sheets with grading feedbacks as oppose to manually marking each paper
- TIMING EXAMS
- can flag students who start exam before designated time/ after stop has been called.
- helps in large testing environments
- evens the playing field between the front and the back of the class
- use algorithm that estimates time spent on problem by dividing time between penstrokes on different pages in half
- start stop box to check
- ANOTO NETWORK
- pen transmits to receiver, receiver transmits to teacher, teacher transmits to TA
- bluetooth lily pad
- real time test monitoring
- give TA's handhelds to monitor student's paper in real time
- have IM communication between student and TA
- LARGE EXAMS
- finite amount of TA's
- anti-cheating/TA distribution.
- TA communication
Idea Selection
Our group chose to apply the Anoto pen/paper technology in taking examinations in order to counter the problems of cheating as well as provide an easier way for teachers to evaluate students’ performance in exams.
The individual proposal of each member in the group were related to each other; they all addressed digital testing in some way shape or form. This set the initial focus of our brainstorming session. At the beginning, the group focused on how digital testing can promote anti-cheating. As ideas poured in, we saw an opportunity to extend our idea so that it included gathering exam statistics that would be useful to our target users. We then started to outline other advantages in using digital pen/paper in examinations: speed up grading, feedbacks to students/teachers, etc. In short, our ideas built on top of each other and this is the reason why we hoped to include everything from our list into our group’s project proposal.
Target Users
The target user group in this case would be classes taking exams in a large dense environment. An example would be the lower division math classes that take their finals in Haas Pavillion. Our project not only aims to help the professor, but also make lives easier for students and TA's; basically anybody who is connected to the exam.
Problem Description
Test Admin
In lower division classes, test admins, such as TA's and GSI's are often vastly outnumbered. Math 1b, traditionally holds its finals in Haas Pavillion. With 4-5 GSI's patrolling the test area, it is next to impossible to truly stop cheating. It also makes it hard for GSI's to answer questions in a quick manner.
Professors
In a large test setting, it is hard to keep track of start/stop times. Even though you might call an end to the test, there may still be students in the back frantically scribbling away. This gives an unfair advantage to certain students.
Additionally, even though a bell curve may tell you how a class did as a whole, it's hard to pinpoint exactly what weaknesses or problems students had on that particular subjects. Right now Math 1b has 174 students enrolled. Imagine trying to analyze 174 different tests for statistical trends.
Students
As a student, you want to have an even playing field when it comes to tests. The location you sit in should not give you an advantage. You shouldn't have to wait precious minutes to get questions answered.
Context
Connectivity
Since this application is meant to be deployed in large classrooms, auditoriums, or even enclosed stadiums like Haas Pavilion, communication between pens and analytical devices will be a significant challenge. With the limited range of each pen's Bluetooth, the development of a reliable and consistent network in the examination room will be very important. It may be more feasible to first deploy a "proof of concept" using a smaller area and fewer number of pens, postponing large-scale implementation.
Reasonable Implementation Cost
It is unlikely, in light of recent tuition increases and system-wide budget cuts, that UCs, or any institution for that matter, has a significant budget to devote to new applications. As such, this application should not be significantly costly above the initial overhead of the Anoto-enabled pens. A solution does not necessarily need to be low-tech, but it should make efficient use of available resources.
Why Anoto
Paper Interface
The use of paper in testing situations has been shown time and time again to more comfortable, and ultimately preferable, to any more electronic examination methods. Whether it is the texture, the physical sense of the writing surface, the high contrast of black ink to white background, or other more or less intangibles, students are more confident taking a test on paper, and thus will perform closer to their potential. Since we do not want to negatively affect student performance, this makes paper an ideal medium for our application.
Additionally, paper is extraordinarily cheap to manufacture in mass quantities, fitting our need to make as little dent in institutional budgets as possible. Printing the grid of dots onto the paper for Anoto-enabled pens to read is also both cheap and fast with a sufficiently high-resolution printer. This is especially beneficial when you consider the class-size this application is intended to eventually serve; printing 300 Anoto-enabled exams will not be significantly more costly than printing 300 of the original exams.
Pen Interface
In the same sense that paper is an optimal medium for test-takers, the pen is the most intuitive tool a student can use to take a test. It is a far better option than typing are otherwise inputting work into some kind of word processor, and it even outshines a tablet interface by eliminating the few configuration interactions necessary on a tablet. With an Anoto-enabled pen, the student need not be aware that they are using anything but an ordinary pen, and so will not be distracted from their exam.
Anoto has also integrated into their pens features that will be very useful in our application. For example, real-time monitoring of what students are writing is essential for the functions of potential cheat detection and e-questioning TAs, GSIs, or professors. Also, our application's testing data regarding time spent per question will be derived partially from Anoto pens' ability to timestamp keystrokes, and even play them back later.
Solution
Our solution comes in 2 parts:
- Statistical analysis of Anoto Feed (Realtime and post test)
- TA distribution/notification system
Statistical Package
The realtime part of this system will use timing algorithms in order to monitor possible cheaters. This part will link up to the TA distribution system in order to try to reduce the number of students cheating. Additionally, by having the professor keying in test begin/end times, they can grade students based on all the material that they wrote before they call an end to the test. So all that frantic scribbling while the TA's are collecting tests amounts to nothing.
The post test analysis will also depend on the timing data stored, but will break down time spent on problems in order to show the professor what subjects the students had problems with.
TA system
This part of our project will work on networking the TA's to the main test admin and the students.
Students will be able to check off certain areas on pages in order to notify the TA's proctoring the test that they have a question. This can be linked to the statistics package and the Anoto page ID in order to record which questions were vaguely worded. But more importantly this allows for TA's to respond to questions rapidly. A student doesn't have to sit around waving his hand like he wants to learn how to fly in order to get a TA's attention.
By keying pen ID's to test locations, the system can notify TA's when students sitting close to one another are working on the same problem. This allows TA's to move observe the test area in a more effecient manner.
Mock System Level Diagram
Mock Professor GUI
Red would indicate potential cheating flags, yellow being students with concerns or questions.
