LoFi-Group:O
From CS160 User Interfaces Fa06
Contents |
Introduction
This project attempts to allow users access to sophisticated game analysis tools while allowing them to play games using pen and paper in the way that they are familiar. There are many games which are better suited for play with pen and paper, but for the ones which are non-trivial the computer provides the only reasonable means for in depth analysis of the entire game tree. For this project we are going to specifically focus on implementing "Dots and Boxes" and "Tic Tac Toe" as our flagship games since they are two of the most common paper based games people play. The only interactive analysis tool we could find with support for both games was the Gamesman suite developed by the GamesCrafters research group here at UC Berkeley. Not only does this suite provide us with many useful game analysis tools, it also gives us the ability to modify and add to the existing code base in order to adapt the software to fit our needs. The best comparable tool for interactive analysis and game play is this program developed at Wisconsin which is text based, only for "Dots and Boxes", and doesn't have nearly as many features as Gamesman. Most existing projects in this realm are aimed at either solving/analyzing the game or playing it, and usually not a combination of the two. This project aims to combine the positive affordances of pen/paper and computers, creating an overall better gaming experience. We conducted the experiment below to get feedback about our first design attempt from potential users of the system.
Mission: In addition to creating a better gaming experience, our major goal is to simplify at every step what is required of the user. We want to make the gameplay on paper mirror the person's normal playing style, except using Anoto pens and paper. We want it to be easy for the user to transfer their game data from the Anoto pen to the analysis tool on the computer. As much as possible we also want to be able to modify the analysis tool to best allow for user interaction in our context. By creating a system that provides a unique set of affordances and is simple to use, we intend to make old school, paper based games more enjoyable for people who still play them in the modern world.
Alex's Role
- Designed and created the first draft of the lo-fi interface.
- Played the "processor" at all of the user interviews.
- Described the prototype in the report below.
Jason's Role
- Acted as observer in experiments.
- Assisted in creating higher quality versions of lo-fi prototypes
- Wrote Method and Results part of write-up.
Michael's Role
- Acted as observer in experiments.
- Assisted in creating higher quality versions of lo-fi prototypes.
- Contributed to results section.
Sean's Role
- Designed and created the first draft of the lo-fi interface.
- Acted as interviewer and greeter in experiments.
- Drafted the introduction and mission statement.
- Researched and discussed possible alternatives to using Gamesman.
Prototype
The lo-fi prototype described in this document was created in two steps. Initially, the interface was sketched out by hand in order to determine which windows and elements were necessary for this prototype. GamesmanWindowFrame.jpg, SidebarElements.jpg, and DialogElements.jpg are the initial sketches upon which the lo-fi prototype was based. The first attempt at the sketches was done in a manner that would have allowed them to be directly cut up and used. However, after considering this for a while, we decided that the interface would be cleaner and more understandable if its elements were constructed using a computer.
The second draft of the interface was much cleaner-looking than the hand-drawn one and was dubbed the "Not-quite-as-lo-fi prototype." This draft used modified screenshots of the actual Gamesman software for the portions of the interface that were a part of Gamesman, and used realistic-looking dialog boxes for the rest. The images Gamesman-lofi1.jpg, Gamesman-lofi2.jpg, and Gamesman-lofi3.jpg show the components of this version of the lo-fi prototype, and this was the version that we presented to users.
The lo-fi prototype had three main components: the Upload File Dialog, the Main Gamesman Window, and the Individual Game Window.
The Upload File Dialog
This is the only part of the interface that was designed from scratch for this prototype. It appears when the user docks their pen and guides the used through the process of uploading the games that they played. The box on the left contains an image of the game that is currently being uploaded (in the lo-fi prototype, this image was generated by the "processor" drawing the game into the window. As such, it was not necessarily a perfect representation of the user's game).
The right side of the dialog then contained tools for selecting the game as which the strokes should be interpreted. The dropdown box labeled #1 contains a list of all the valid games that our uploader understands (in this case, Tic-Tac-Toe and Dots And Boxes). When the user clicks on it, the processor presents them with a dropdown list, and then overlays a piece of paper on which their choice is written over the dropdown box. Once the user makes a selection, the area under #2 is populated with a list of options selectable with a radio button that can be used to specify the rules by which the game was played (for example, if the user picks Tic-Tac-Toe in the dropdown box, this area is populated by the single option, "What was the win condition for the game?" and the choices "Standard" and "Misere"). Radio buttons are selected using a tiny square of paper which is colored in to look like a radio button. The area marked #3 is a text box in which the user can enter a filename under which they want to save the game. Like the dropdown box in #1, this textbox is overlaid with a slip of paper containing the user-specified name.
At the bottom of the window, there is a line telling the user which game he was uploading: "Game ____ of ____." This is filled in by the processor dynamically. When all games have been uploaded, a dialog box pops up that says, "Done uploading games. Launch Gamesman now?" and presents the user with a Yes/No decision.
The Main Gamesman Window
The Main Gamesman Window is the first thing the user sees upon starting Gamesman. The center of the screen displays a short description of Gamesman, and the bar at the left side of the screen allows the user to select a game. Again, only Tic-Tac-Toe and Dots And Boxes are supported in the low-fi prototype. When the user selects one of them, the central display changes to a description of the game. From there, the user can click on Play GUI to launch the game.
There are some other features in the actual Gamesman client that do not pertain to our interface, but these were removed in order to simplify the lo-fi prototype.
The Individual Game Window
This window loads when the user clicks Play GUI from the Main Gamesman Window. There are several options along the top, but all except New Game and Load Game are grayed out as they don't pertain to our interface. When the user clicks New Game, the box in the middle is filled with an empty game board. When the user clicks Load Game, an Open File dialog appears which is populated by all of the games uploaded by the user previously. The user selects one and clicks Open to load it. The result of this is identical to the results of clicking New Game, except that the game board starts off in whatever state the uploaded game was at.
The empty game board is represented by a piece of paper with the game board drawn upon it. As players make moves, the processor marks these moves with cutout X's and O's, as well as cutout line segments for Dots And Boxes. If the user clicks undo, the processor removes the last element that was added to the board.
The radio buttons at the bottom can be used to toggle the sidebar at the left. The default sidebar shows the rules of the game and the objective. If the user toggles it to Value History, the sidebar switches to a graph of which player had the advantage at each point in the game.
When the user is finished, he can click the Quit button in the top right corner to quit the game.
Method
Participants
The participants of our lo-fi prototype experiment were chosen from our two target test groups. One of the participants was a member of the GamesCrafters research group and was fairly familiar with the GAMESMAN system. They were chosen to participate in order to get their opinion on the pen-and-paper interactions with the GAMESMAN system. The other participants were taken from the "regular gamer" target group, and were chosen from a student group meeting on campus who were gathering to play games, both of the board game and the video game variety. They were chosen to participate in the experiment due to their familiarity and love of gameplaying.
Environment
The environment in which this prototype experiment was carried out was similar to those in which the participants would normally play the pen-and-paper type games that we will be implementing in our product. The experiment was carried out simply on a table in a classroom where the participants were meeting on campus. In the case of the GamesCrafters participant, the experiment took place at an alcove on the 6th floor of Soda Hall, which is the floor where GamesCrafters meetings generally take place. In the case of the on-campus student group participants, we simply attended one of their meetings and asked to take some members aside in order to carry out our experiments.
Tasks
In our experiment, we asked our participants to carry out the six tasks of our product, as they were all fairly tied-in with each other and one task often led straight into the next task. The six tasks asked of each of our participants was, in order: Starting a game, Playing a game, Finishing a game, Uploading a game, Loading a game into GAMESMAN, and Using the Visual Value History. The participants were asked to perform all these tasks, as it is an accurate representation of all the tasks that users would need to do in order to reap the benefits of our product, namely having the benefits, comfort, and convenience of playing pen-and-paper based games on actual pen and paper and also having the powerful analysis tools that are found in the GAMESMAN system.
Starting a game: Though simple to do with an understanding of the rules and familiarity of the game, this task is essential as it provides the user with the ability to start collecting the raw data which he or she will be uploading to the GAMESMAN system.
Playing a game: Simliar to starting a game, this task is extremely important, as it contains the data that will later be analyzed by the Anoto pen and the GAMESMAN system.
Finishing a game: This task is one that is unique to our product. In order to signify that a game is complete, the game will be circled before moving on to the next game. This is important because it separates the games so that the upload interface can distinguish between different games and parse the stroke data correctly.
Uploading a game: This task is important to the system, as it provides the "bridge" interface that converts the data on the digital paper into a format that can be understood and analyzed by GAMESMAN properly. This is probably the most intensive task, as it involves the user to remember each game he played and enter some data on it, including rule variations that were used during the game and offering a filename.
Loading a game into GAMESMAN: The user launches GAMESMAN and uses the open file command to load the game that he or she uploaded at an earlier state.
Using the Visual Value History: This task is the goal of the tasks we presented our participant using the Anoto digital pen and paper interface, in which the game that was played on paper is analyzed using the VVH tool of GAMESMAN. The game can now be analyzed, good and bad moves can be identified, and strategies can be formed.
Procedure
For our experiments, we divided our team into four roles: the greeter, the computer/processor, the facilitator, and the observer. The greeter usually doubled as a second observer, and the facilitator would often be the opponent that the participant faced when playing the games, as the two games we played with the participants (Tic-Tac-Toe and Dots and Boxes) required two people to play.
Upon meeting the participant, the greeter introduced himself to the participant and gave some background to the purpose of our lo-fi proptotype experiment, which included telling participants about the class (CS160), the product that we were currently in the middle of designing, and also some information about the GamesCrafters research group and the GAMESMAN system and how our product tied in with their work. The greeter also presented the participant with the consent form, informing the partcipant that his information would only be used in the context of improving the interface design of our product and that his identity would remain anonymous. Upon signing the consent form, we would continue with the experiment.
The facilitator would first perform an example task to demonstrate how to interact with the paper interface. This demonstration task involved interacting with a lo-fi paper replica of the GAMESMAN system, and the facilitator demonstrated how to play a game of Tic-Tac-Toe on the GAMESMAN paper replica. Though it was easy enough to explain to the participant that touching the paper in an area was equivalent to clicking on that area with a mouse, demonstrating an entire task from start to finish gave the participant a chance to see how interacting with the lo-fi prototype worked, minimizing errors in understanding how to use the paper interface. During this task, the facilitator would choose to play a game of Tic-Tac-Toe on the GAMESMAN system against the computer. The team member playing the role of the computer/processor would assume the role of the opponent against the facilitator in this case.
After demonstrating the playing Tic-Tac-Toe on GAMESMAN task, the facilitator would then ask the participant to begin completing his tasks, starting with simply playing a game and culminating with opening an uploaded game in GAMESMAN. First, the participant played a series of games of his choice against the facilitator. It was also explained to the participant that in order to signify that a game was finished, a user would circle the game before moving on to the next one. The participant continued to play games (at least two) with the facilitator until he decided that he had enough games, at which point he was instructed to "dock" the pen (usually this was done by simply handing the pen he was using back to the facilitator). Upon docking the pen, the participant would move on to interacting with the Anoto pen and paper interface to upload pen data in game format.
The participant would be presented with an interface in which he could upload the games that he just played onto his computer. The computer/processor would manipulate the lo-fi paper interface in order to reflect the changes and options that the participant clicked on and selected. The participant would be presented with a lo-fi bit-map rendering of the first game that he played and the interface would ask him to select the type of game he played, which rule variations he played with on the game, and also what filename he would like to give the game. The participant was also given the option to skip any game that he wished to not upload to the system. After loading (or skipping) through all the games, the participant was presented with an alert window stating that he hd uploaded all the games, at which point he would be given the option of launching GAMESMAN to load the game(s) that he just saved and analyzing them with Visual Value History.
After uploading the game to GAMESMAN, using the VVH tool, and possibly playing around with some of the undo move features of GAMESMAN, the participant was debriefed on the experiment and asked some follow-up questions, including what he liked and disliked about the interface and what we could do to improve the interface. After gathering some feedback, the participant was thanked for his time and we discussed the results of the experiment and what needed changing in the interface.
Test Measures
In our experiments, we were mainly looking out for any mistakes that the participants made in interacting with our lo-fi prototypes. Specifically, we were looking for moments in which the participant assumes that an action he did would have a certain result, but in fact a different result would occur. Looking out for these moments was critical to improving our interface, as any of these occurences meant that not only was something in our interface not clear, but the participants were reading it as meaning something different. Eliminating any chance for these moments to occur was critical in creating a smooth interface that would be easy to understand and carry out a user's intentions.
In addition, we were simply looking out for any moments of confusion that arose from our participants, as we asked them to think aloud as they were carrying out their tasks, especially if there were any moments that it was not clear what needed to be done next. Participants were also asked to point out anything that they felt was inconvenient, nonsensical, or any suggestions that they had for general improvement, as well as any positive feedback they had on the system. We generally did not look for any bottom-line data, other than perhaps the frequency at which concerns were brought up (frequent concerns meant that that particular aspect of the interface would need more work, non-frequent concerns meant that that particular aspect of the interface is probably fine). The test measures we mostly watched out for had to do with the severity and urgency of our participants' feedback from their time interacting with our lo-fi prototype, as well as our own observations on anything that our participants may have overlooked or misunderstood.
Results
First experiment
The first experiment involved the participant from the GamesCrafters research group. After finishing the tasks involving the digital paper (starting the game, playing the game, finishing the game), the user was presented with the upload game interface. He was able to identify and correctly carry out the first step, but upon looking at the rule variations for the Dots and Boxes games (step 2), he was confused over what a board size of "2x2" and "3x3" meant. There was confusion over whether "3x3" meant a board size of 3 dots by 3 dots or 3 squares by 3 squares (this incident was deemed a rating of 4). Upon clearing up the confusion, the user was able to finish up the task on his own.
Overall, this user was able to complete most of the tasks fairly easily, due to his familiarity with the GAMESMAN system. However, there were some concerns brought up with our interface. Though the user did not really notice it due to only uploading two games, we noticed that there was some redundancy in having the user have to repick his choices for uploading each game, particularly if the user played the same game each time with the same rule variants (which is often the case in real life). We decided that it would be convenient if the system were to "remember" the previous game choices while uploading each game, so that the user could have the option of picking the same rules without having to click through all the pull down menus and radio buttons each time. This incident was rated a severity of 3. In addition, the user commented on the desire to have the option of adding text comments to each uploaded game (rating of 2).
Though the participant asked many questions throughout the experiment and had some concerns, he was able to complete his tasks fairly smoothly and with little difficulty. He even enjoyed the smoothness of the paper interface.
Second experiment
The second experiment involved a participant from the game players group meeting on campus. Overall, this participant had many of the same concerns as the participant in the first experiment, including confusion over the board size of dots and boxes. The participant also asked many of the same questions, such as what would happen if a user made a mistake, such as entering the wrong rule variations when trying to upload a game (in this case, either the interface would return an error message, stating that it was not able to recreate the game based on the user input and the pen data, or the game would be uploaded, but incorrectly and probably a differnet outcome). Overall, though, the experiment went about as smoothly as the first experiment and few major issues were brought up.
Third experiment
The third experiment involved another participant from the game players group meeting on campus. Though it went fairly smoothly as well, there were more concerns and questions brought up in this experiment than in previous experiments. This particular user had some concerns with some of the terminology used in the game variations, particularly the use of the word "misere" (which means a win condition in which a player forces the opponent to take the "winning" move). This was rated an incident rating of 5, and led us to strive to clarify the terminology used on the interface. In addition, the user expressed his concern that there was not a help button and that there was not a "back" button on the upload interface where the user could return to upload a game he had accidentally skipped. This was also rated 5, as it was an important need that was overlooked.
The user attempted to upload a game by pressing the upload button without entering any information first. It was not clear that this was not an option, and we noted that the upload button should be "grayed out" if sufficient information had not been entered (once again, a rating of 5, as uploading without gathering all the information may cause erroneous uploads). In addition, the user was somewhat confused as to some of the labeling conventions of GAMESMAN. In the game of Dots and Boxes, the player usually "claims" a box by writing his or her initial in the box. However, in GAMESMAN, the boxes are automatically filled in with Xs and Os, in which the X is the player that made the first move. The participant had some confusion as to who was X and who was O, leading us to explain that X was the player that went first. We decided that there should be some text during the upload game phase that explains GAMESMAN's labeling conventions. This incident was given a rating of 4.
This experiment was probably the most helpful, as it revealed that we may have assumed some prior knowledge that users may not have had.
Discussion
What we learned from conducting our experiments was that it is dangerous to assume that the user has some prior knowledge that is essential in using the interface. A big example of this was in our upload interface, where we asked users to enter the rule variations that were used in the game that they played. For Dots and Boxes, there was confusion over whether "3x3" meant the number of boxes or the number of dots on each side of the board. Also, for both games, there was confusion in each of the games played over what the term "misere" meant. We realized that although we may be familiar with some of these terms, the typical user may not, and it is important to spell out in explicit terms what everything means, so that there is no conufsion over what the user is doing.
Several issues/questions arose about how the game is played on paper vs within GAMESMAN. For example, in Dots and Boxes, filling in the box with a player's initial can be done out of order with the corresponding moves (such as taking several boxes in a row), or may be skipped entirely at the end of the game. For Tic-Tac-Toe, it may be the case that the O player goes first on paper, while GAMESMAN does not allow that option. It's clear that our system must be flexible in allowing the players the freedom to play however they want, but be careful to point out possible pitfalls or differences when the computer interface is used.
We also learned that users may either skip some steps in working with the interface or may attempt to do something different than what the user needs to do, just to see what may happen. We realized that in order to minimize errors in uploading data, we must take every measure to prevent the user from trying to upload the data before all necessary information has been gathered. This was explicitly demonstrated in our third experiment, when the participant tried to simply hit the upload button before he had entered the game type, rule variations, or file name. Therefore, we decided that it is important to explicitly gray out the upload button, making it unclickable until all the necessary information has been gathered. This will eliminate the possibility of erroneous uploads of data, which will only confuse the user when working with our system and the GAMESMAN system.
The results we gathered will not change the flow of our interface, but we will make some changes that may seem minor from a cosmetic point of view but are actually important and even critical to imporving the usability of our interface. These changes include the ones mentioned above in the discussion, the inclusion of a help and back button in our upload interface, a feature that saves the choices from the last game (so that in the common event of playing the same game many times, the user will not have to reclick each of the choices for each game), and some explicit text explaining the labeling conventions of GAMESMAN and game variations. Also, within the GAMESMAN interface, a redo button could be added to help with traversing the game flow in both directions. These seemingly minor changes should go a long way in creating a smoother flow and ease of usability of our interface.
Since we are exploring relatively new territory, our experiment may not have been able to reveal any extra features that users may want that we have not yet included. Perhaps only those really familiar with the GAMESMAN system would be qualified to answer a question like that. However, extra features will in our interface will largely be dependent on the expansion of the GAMESMAN research project, so it may be the case that users will discover that they want to see more features implemented in our project as GAMESMAN grows.
Lo-Fi Interface Elements
| Initial element sketches: | ||
 |  |  |
| Computer-drawn elements for printing: | ||
 |  |  |
Interview Images
 |  |  |  | |
 |  |  |  |
