From CS160 User Interfaces Fa06

Final Report (60 points) Your report should serve as a stand alone description of your project and should therefore summarize some things you have presented in previous reports. It should also go into detail on a few new aspects for your project (new since the interactive prototype). We will be grading the overall report as a whole but place most emphasis on 1) how your UI changed over the course of the project based on your evaluations, 2) which evaluation techniques were most effective and 3) the description of the final UI.

The report should follow this outline with separate sections for the top-level items.

Contents 1 Each team member’s name and role in this assignment 1.1 Bowen Li 1.2 Yimin Yao 1.3 Jonathan Yen 1.4 David Hoffman 2 Problem and solution overview (1 paragraph) 3 Target user group (1 paragraph) 4 Tasks (1/2 page) 4.1 Hard Task: Create & Import 4.2 Medium Task: Organization: Move/Set familiarity level 4.3 Easy Task: Study Flashcards 5 Design Evolution (2 pages + sketches & screen shots) 5.1 Initial Sketch 5.2 Low-Fi Testing 5.3 Interactive Prototype 5.4 Pilot Study 5.5 Modifications since Pilot 5.6 Most Valuable Evaluation Technique 6 Final Interface (4 pages + screen shots- reference figures!) 6.1 Describe the final UI design 6.1.1 Import Tab (fig.4a, 5a) 6.1.2 Browse Tab (fig.5b-c) 6.1.2.1 Comfort Level 6.1.2.2 Folder 6.1.2.3 Features Displaced 6.1.3 Study Tab (fig.3e, 4d) 6.1.3.1 Presentation Order 6.1.3.2 Dual sided studying 6.1.4 Quiz Mode (fig.5d-f) 6.2 Implementation 6.3 What was left unimplemented 7 Appendix

Each team member’s name and role in this assignment

Bowen Li

Bowen authored the software responsible for the quiz tab of the software. This consisted of coding both the user interface and the engine governing presentation order and the distractor choices.

Yimin Yao

Yimin was in charge of modifications on the import panel as well as resolving the reloading cards issue to eliminate Wizard of Oz step. She also worked on the presentation and the Design Evolution section, along with some last minute bug checking.

Jonathan Yen

Jonathan worked on the browse tab and improving the flow of user actions as they navigate through the software. He also worked on the problem of making the menu selection a more intuitive process for the user.

David Hoffman

David was responsible for presenting the background information for the final prototype. This material included, the target audience, the tasks, and the goals of the software solution. He also assisted in planning out some of the changes to the software.

Problem and solution overview (1 paragraph)

The goal of the BluJay Flashcard software is to solve some of the problems with studying flashcards. Our main maxim for this project was to avoid any possible harm. Studying for an exam is a critical task, and as such, must have an ironclad contingency plan. We decided early on that flashcard studying was too critical to entrust fully to a computer, and that often flashcards are used in transit and right before an exam. The potential for problems, the cumbersome load of computing equipment, and clumsy traditional input for non-text information, makes a computer a poor replacement for paper flashcards. We found that use of a digital-paper interface provided an elegant solution to many of the complications of flashcard usage. The paper side of this interface retains the robust qualities and free form construction of traditional flashcards. The digital interface allows for complex organization of flashcards across multiple subjects and for varying levels of difficulty. The software package allows the user to rapidly browse through their library of flashcards and to organize them by directory. It also allows them to study the cards in randomized order, and specify what cards they would like to study, i.e. hard or easy. The digital flashcards also allow for the capability to show the front and back of the flashcard simultaneously, which would not be physically possible.

Target user group (1 paragraph)

We are targeting this software towards experienced flashcard users, with a special emphasis on users who are using flashcards to study for multiple courses and/or multiple exams. One of the greatest strengths of this software is its organizational feature, allowing people to manage huge volumes of flashcards belonging to different classes. This software, while applicable to any memorization chore, is optimal for studying information requiring non-ASCII material. This could include foreign languages, math and chemistry. Users studying this material with complex figures on the flashcards would be unlikely to utilize an input interface other than pen on paper.

Tasks (1/2 page)

We have defined and used the following three tasks for testing our interfaces:

Hard Task: Create & Import

Task Summary: On the import panel, the user selects subjects of interest and his/her flipping style, then uses an Anoto pen to create and import a sample set of flashcards. After the completion of each flash card, the user must tap the 'DONE' box using Anoto pen or click the 'DONE' button on the import screen to register the card into the flash card library.

Reason for choosing this task: This task was chosen because it is the most critical aspect of the software, since all other functionalities of our software depend on the existence of a flash card library. We designated this task as hard because users must learn to use the Anoto system and deal with any technical difficulties inherent in such a system. In addition, the user must consider factors such as flipping orientations that they normally would otherwise not consider when creating paper flashcards. We have minimized the difficulty of this task by making the flashcard creation process as similar to traditional flashcard production as possible.

Medium Task: Organization: Move/Set familiarity level

Task Summary: In the browse tab, the user views and organizes the flashcards in the library. Most of the cards are already in the correct folder. However, whenever the user discovers misplaced cards, they must be moved into the folder they correctly belong in. Lastly, as the user browses through a particular subject folder, he/she should be able to set the comfort level for particular cards to familiar (or their appropriate comfort level) because he/she has learned them well.

Reason for choosing this task: The task was chosen because organization of flashcards from different subjects is an important feature of our flashcard system; operations such as moving and deleting cards are critical in managing a flashcard library. The task is considered medium difficulty because although all the operations are very straightforward, it requires some initial learning of the interface to master all the features of this panel as well as learning the flash card directory structure.

Easy Task: Study Flashcards

Task Summary: In the study panel, the user studies a subject by selecting a folder and the desired familiarity level(s) he/she wants to study, then traverses through the cards with a few basic buttons such as previous, next, and flip(to see the other side). In addition, users can modify difficulty levels of cards as they master new information; they can also choose the mode of display so that the reverse side or both sides can be studied first.

Reason for choosing this task: This is intended to be the easiest task because we want the users to fully focus on studying the material and not be distracted by the software. Thus we have tried to enable users to study from the flashcards as naturally as possible while adding a few useful options. No operation will result in error and all of them are easily reversible if the user changes their mind about their study options. This task is arguably of equal importance as importation because this software would provide no value if the cards could not be used on the computer. While the effectiveness or ease of studying with software is difficult to study in a contrived situation, this task is representative of a study session, with the user performing all the actions they would need to do to study the cards effectively.

Design Evolution (2 pages + sketches & screen shots)

Initial Sketch

• Overall

At the initial stage of the design process, we determined our target users, and determined key features of our software based on contextual inquiry. A rough sketch of the framework with four tab panels is produced, and how each functionality will be carried out is proposed.

• Paper Interface

- No detailed paper interface is sketched at this stage due to the lack of knowledge about Anoto system

• Import Panel

- No detailed Import panel is sketched either due to the lack of knowledge about Anoto system

• Browse Panel (see fig.1a)

- Thumbnails of flashcards can be viewed by clicking their corresponding subject folder

- Buttons are available to delete, move, select, set comfort level, and begin to Study or Quiz selected cards

• Study Panel (see fig.1b)

- Study Panel is accessed through Browse Tab by selecting the cards of interest, and click study

- Study cards one by one with film strip at bottom, and options to set comfort level and study from either or both sides

• Quiz Panel (see fig.1c & 1d)

- Quiz is presented as a set of multiple choice questions with one front side and four back sides as choices

- Result would be shown at the end of the quiz

fig.1a	fig.1b
fig.1c	fig.1d

Low-Fi Testing

• Overall

The Low-Fidelity Prototype used for experiments is developed using regular paper and pens. A typical screen of the prototype consists of four frames (fig.2a).

1. Conventional toolbar (on the top): File, Help, etc

2. Library (on the left): A column of directories for navigating through all the subjects of flashcards that have been entered into the system.

3. Main frame (biggest region on the screen extending to lower right corner): Region where flash cards are displayed individually for studying, by subject groups for browsing, or shown in a question format for quizzing.

4. “Mode” Tabs (above main frame): allow users to switch from one task to another (for example, from studying to taking a quiz)

• Paper Interface

Regular paper is used for this stage of prototyping (fig.2b). Users are asked to make cards using regular pen, as soon as the user inserts the pen into the “pen holder”, an import screen is displayed. We put their cards on our interface to show that they are in our database.

• Import Panel (fig.2c)

In this stage we assumed that we would be using batched import. The import screen prompts the user about the subject of the cards to be made and the format in which they want to import the flashcard such as the way they flip cards(this is important because we want the correct orientation of the card in the library). The import screen will also inform user how many cards are being imported.

• Browse Panel (fig.2d)

This screen shows the front of each card as a thumbnail, with a checkbox next to each card to indicate whether or not the card will be manipulated. All the functions remained the same since no user testing has been conducted yet.

• Study Panel (fig.2e-g)

This screen allows the user to select a desired group of cards, and specify the manner in which they want to study the cards. (fig.2e) By choosing 'sequential' study, the user can navigate by clicking on the arrows under the card, or by clicking on a card in the "filmstrip" at the bottom of the screen (fig.2f). Also provides some options for studying and changing the rating for the current card; if the 'random' study mode is chosen (fig.2g), the "filmstrip" is covered (invisible) because it has no meaningful function.

• Quiz Panel

The quiz screen generates a multiple-choice quiz based on the card selection. Each quiz question contains the front of one card, then gives 4 choices for the user to pick from. After clicking on a card, the user may go on to the next card or quit the quiz. Results of the quiz is shown at the end.

fig.2a	fig.2b
fig.2c	fig.2d
fig.2e	fig.2f
fig.2g	fig.2g
fig.2g	fig.2g

Interactive Prototype

• Overall

In the interactive prototype, we implemented the major framework of the software application, featuring all the selection menus and buttons; however, since we were not able to get streaming working, most of the functionalities were substituted with a Wizard of Oz technique using fake flashcards.

One other large modification to the interface was the elimination of the left navigation frame on import, study and the quiz panel, since it does not have significant function while occupying large area of the interface. We also decided that user would study or quiz by directly clicking those tabs because users from lo-fi testing expressed the concerns of possible confusion by automatically jumping tabs (jumping from browse to quiz/study).

• Paper Interface

The paper interface has been greatly revised at this stage. Anoto patterns have been used to print the flashcards. To organize the cards in uniform orientation, "front" and "back" letters are added to the card (fig.3a) because we observed during lo-fi prototype study that users would often begin to write on either side of the cards at random. In addition, the "done" button is added because we were forced to use streaming mode for data input due to technical limitations. After the user completes writing on front side and back side, the "done" button is clicked to initiate the creation of the flashcards (JPEGs).

• Import Panel

The import panel's look was implemented as previously designed (fig.3b) except that the left navigation panel was omitted. Since the streaming was not implemented, Wizard of Oz methods had to be used for streaming import screen(fig.3c) once the 'start import' button has been clicked.

• Browse Panel

In this panel, card selection and drop down menus were mostly implemented as the original design (fig.3d), however all the cards were preset images rather real flashcards. Some interviewees during the lo-fi prototyping was confused by the set comfort level immediately followed the "move" button. Thus we moved the "move" button to the right end of the interface, and left enough space between "set comfort level" and "move" panels.

• Study Panel

The study panel also had its major framework set up (fig.3e). One major modification was that the study panel was accessed by directly clicking its tab rather than going to browse panel. The interactive interface was able to traverse through the fake cards by clicking the next button.

• Quiz Panel

The Quiz feature was not implemented in this version of the prototype. One reason is that we want to make sure that the user has found the basic program features usable before adding high-level program features. Furthermore, during the early qualitative evaluation of this product, we did not want our users to get hung up on the details of the quiz. Without a large database of cards, which has come from importing many flashcards, the multiple-choice portion of the quiz will seem highly repetitive.

fig.3a	fig.3b
fig.3c	fig.3d
fig.3e

Pilot Study

• Overall

The most changes since the interactive prototype are from the completion of implementation of various features. Most importantly the streaming import, the organizational functions, and integrated study panel. The only Wizard of Oz part was the flashcard library reloading between each of the panels by reopening the application.

• Paper Interface

No obvious changes were made; its size was slightly adjusted for more convenient printing.

• Import Panel

In the fully functionality import panel, user can now create flash cards in either flipping orientation using Anoto pen (fig.4a-b). Although batched import was an option, we decided to stick with streaming because real time feedback for strokes would greatly reduce the error rate for creating flashcard. In addition, the images for flipping instructions were updated to clearer illustration because one user from the low-fi testing complained about the confusing arrow directions.

• Browse Panel

All the buttons on the browse panel were fully implemented (fig.4c). Users can now view cards by subject by clicking the folder icon on the left panel. they can also move or delete cards as well as setting comfort levels for selected cards.

• Study Panel

The study panel now supports the option of studying from either side of the flash card or both sides simultaneously (fig.4d). A new button was added to allow user to go back to main study panel without finishing studying the list, because users would often times change their mind or make a mistake in selecting the cards of interest. The study navigation was based on navigating a photo browsing program.

• Quiz Panel

The quiz panel was not implemented for this round of testing.

fig.4a	fig.4b
fig.4c	fig.4d

Modifications since Pilot

• Paper Interface

The backside of the flash card is marked by horizontal lines across, because several users mistakenly started importing on the back side during the pilot study.

• Import Panel

'Done' button was disabled before arrival of any written content to prevent blank cards, which were a problem in pilot study when the 'done' button is hit multiple times. Also, an electronic version of the 'done' box is added at corresponding position on the software interface to allow software triggered flashcard creation in case of tabbing difficulties(the difficulties in the pilot study were that sometimes it would take many taps before the click in the small done box would register) (fig.5a).

• Browse Panel

The drop down menu was replaced by scrolled menu with some text instruction because many pilot study users were confused over the buttons due to the improper display of the drop down menu. In this implementation, thumbnail images of the flashcards were used instead of the full sized cards so that users could browse through more cards at once. Users can use a scroll bar to adjust the size of the thumbnails; this feature was suggested by a volunteer during the pilot study. (fig.5b-c)

• Study Panel

Problems with setting the familiarity level while traversing through the cards are now fixed. Before the commands were present to change the familiarity level, but changes were not saved. Now, those changes are recorded in the flashcard file.

• Quiz Panel

The quiz panel now has its basic quiz features implemented. The user can now select desired folder, and take a quick multiple choice test and view their score and his/her performance on each question at the summary page. (fig.5d-f)

fig.5a	fig.5b
fig.5c	fig.5d
fig.5e	fig.5f

Most Valuable Evaluation Technique

Both the Lo-Fi prototype user testing and the Pilot Study were very valuable to the evaluation of our prototype.

The Lo-Fi prototype uses paper cutouts to mimic every user operation and it led us to consider many aspects of the software which we had not expected to encounter. The user feedback helped us understand the values of each feature of the software and enabled us to rough out a more cohesive design. The pace of the testing was such that users would actually take the time to say what they were doing, which provided us with a lot of insight into users' desires and needs in studying with flashcard system.

While the Lo-Fi is good for building up a overall framework and major features, the Pilot Study dives much more into the details of the system's components. With the much more completely implemented prototype, we are able to provide users with real time feedback rather than the delayed effect in the "human computer". The Pilot Study allows us to examine the design of certain features more closely and quantitatively. The implemented buttons enable users to explore the functionalities on the panel in order to complete the tasks, thus providing us with information on learning rate of the interface.

In summary, Lo-Fi prototype testing helped us to make modifications in the overall framework and functionality, while the Pilot Study was very valuable in determining the specific presentation/layout of those functions.

Final Interface (4 pages + screen shots- reference figures!)

Describe the final UI design

At this stage of implementation we have fully implemented the most critical features to make this a viable and usable study tool. (Refer to figure set 5 for most modified parts.)

Import Tab (fig.4a, 5a)

The import function is fully implemented and allows a user to use an Anoto Pen to write on our custom flashcards printed with Anoto pattern. Their strokes on the cards are streamed to the software so that the paper flashcards are mirrored on the screen. This allows the user to correct any problems they may be having with the pen, or the settings they used when setting up the import. Furthermore, from the import setup screen the user can select what directory they would like the flashcards to go to as they write them, and specify their flipping style. This is a critical setting because if the user likes to flip the card in bottom to top method, the flashcard content would appear upside down in the software. The paper interface contains a small box on the back of the flashcard, so that when they have finished the card, they tap the done button and the card will be numbered and saved. The computer will clear the import preview window for the next card. If the user is not comfortable with tapping the done button, then he/she can click the electronic done button on the import screen instead to obtain the same result.

Browse Tab (fig.5b-c)

This tab shows a list of folders and flashcards that are in those folders on the left side of the window. From this folder list, the user can select what folder they wish to browse. In this context, by browse we mean to preview all the flashcards in that folder. The preview consists of the front of each flashcard with a check box in the lower right hand corner of the flashcard. From this window the user can select individual flashcards, or can use the "select all/none" buttons. Once selected, the user can make several changes to the flashcard. In addition, we have added a resize bar that allows the user to adjust the thumbnail size of the flashcards. This allows the user to show more or less flashcards on the screen to suit personal preference.

Comfort Level

One of the menus allows users to specify their comfort level with the selected cards. With the paper interface, the user might make separate piles of flashcards to concentrate their studying on the cards they are most unfamiliar with. The comfort level menu is a digital solution to the multiple piles of paper flashcards. Tagging each card with the user comfort level allows the user to specify which "pile" he would like to study from later on and be a tool to keep track of which cards remain a problem.

Folder

This menu allows the user to correct an error he/she may have made during the import procedure. The import procedure will save all the flashcards to a directory specified at start up. While this is normally sufficient, we wanted this browse function to be especially powerful for users who have numerous classes and could potentially mix up their flashcards. This change folder menu allows the user to move a flashcard to a different directory.

Features Displaced

Our original concept of the browse tab included buttons to manipulate flashcard orientation and sides. This would allow a user to correct a card that was upside-down, or have the front and back reversed. We decided that adding these buttons to this window would only clutter the interface and provide features that the users would be unlikely to need while they had the revised import screen. With streaming pen data going into the import screen, any error in the pen data would be discovered earlier and would not make it to the browse screen. Thus orientation options were removed.

Study Tab (fig.3e, 4d)

The study tab is the mode of the program where we expect users to be spending most of their time. This will be where they can click through their flashcards and attempt to memorize the contents. We wanted to make this as simple as possible so that users are not distracted by any of the software bells and whistles. Thus, we chose to make this window share many of the features that users are familiar with, photo slideshow program controls. Thus we have a button to advance and to go to the previous flashcard. There is also an added button to flip the flashcard. This is meant to follow the proven paper studying analogy as much as possible. There is also a radio button on the side to flip the deck, so that when each new flashcard is presented it will show the opposite face of the card, and pressing the flip button will show the front.

We did diverge from the paper studying method in two ways.

Presentation Order

One user complaint about paper flashcards is that they are presented in order. Our interviewees said that they tended to learn the first card in the deck best and would learn the cards buried deep in the deck poorly or more slowly. They would often have some organization to the flashcard deck and be hesitant to change the order. This could be because they have segregated the cards by difficulty, alphabetical order, or some course specific breakdown. With our software we allow the cards to be kept in order or in directories but be presented at random. We hope that if we continued to test this software that the randomization would greatly aid people in memorizing the flashcards uniformly.

Dual sided studying

The radio buttons on the right show three main program modes. Two of them are conventional flashcards but showing initially either the front or the back. The third button is a dual sided study feature which is not physically possible with the paper interface. This mode will show the front and back of the card simultaneously. Since no flashcard user has ever studied in this capacity, it is still unclear how effective this will be at helping them to memorize the content. However, we feel that this is a feature which will, at least in some circumstances, be very useful and makes strong use of one of the benefits of having a digital paper interface.

Quiz Mode (fig.5d-f)

This is a part of the program which we have implemented late in this project and have not had the opportunity to test it with users, or seek user feedback. This is a feature which we were interested in early on because we believed it would enable our users to use the computer to evaluate their own knowledge. The goal was to have the computer act as a friend and patiently lead the user to remembering the correct answer. We facilitated this with a multiple choice paradigm. The software shows the front face of the card and asks the user to match it with the correct reverse side. The reverse side is displayed along with three other randomly chosen reverse faces from other cards in that group. The distractors are chosen from the group to provide the most plausible and attractive solutions. This is an important distinction because it would not be a reasonable choice to answer a vocabulary question with a math equation.

Implementation

Implementing much of our user interface was a significant challenge. The primary challenge was to effectively collect and store the stroke data. While we had originally wanted to do a batch solution, for a variety of reasons this proved too difficult and we adopted the streaming approach. The streaming approach wound up having many advantages which we were able to take advantage of.

Another particular challenging aspect to implementing our design was getting Java to refresh when something had changed. In our original pilot testing we needed to do Wizard of Oz intervention to reload the software or switch back and forth to a window so that it would be updated with the latest information.

In addition, we had some struggle over formatting the displayed images such as resizing and position. It was difficult to position them properly for aesthetics of the interface, but we did manage to place them close to where we wanted in the end.

What was left unimplemented

What was left out and why

Originally, we had put in stubs for a global menu system at the top, ie “File,” “Edit,” “Help,” etc. but because our program is very self contained, there wasn’t much use for the menus. All of the actions are built into windows through buttons and other GUI elements. Although we could have added functionality to the menus, they would feel contrived an unnecessary.

We did not implement the feature to study by being exposed to less familiar words more often. This mode required changing the paradigm of the study section and developing a completely new algorithm for actually intelligently pulling up the correct cards to study. Since we are not actually doing a study to see if this method is more effective in study, it was left out. It is behaviorally very similar to the “random” mode in the study section from a user’s perspective, so it should not affect the interface design at all.

Although surface-level details are important to any program, we did not change the “look and feel” for our program beyond the provided defaults. The Java implementation and behavior of how the program looks with respect to color, style, etc. was not obvious, and it changed from system to system. Since our application is used mostly for work and not for entertainment, we feel that the style of the program was not essential to the final design.

During the low fidelity interviews, some of the interviewees mentioned that it may be useful in the quiz “review” section to be able to link back to the original question to see what they had chosen. We chose not to implement this feature because we felt that it would require storing too much information (#questions * #choices/question) for the review section. Instead, as a compromise, in the quiz review we show what the original question was, what the user chose, and what the correct answer was. This essentially encapsulates most of the important information that a user would like to see through linking back to the original page.

In the import screen there is an option to create a new ‘subject’ which creates a new folder to put flash cards into. This option is functional, but has not been fully tested with all the end cases. Additionally, there is no way to do this from the ‘browse’ section.

Any wizard of oz techniques that are required to make it work
We have very minor portions that require the wizard of Oz. One of these sections is in the browse section, sometimes ‘refresh’ does not behave as desired and an additional click or resize is required for the refresh to take place. This is supposed to be transparent to the end user.

Also, as mentioned in the previous section, the “show less familiar words more often” feature is not implemented. Functionally, this means that we may have to WOz the fact that a user may see “card C” instead of “card B,” but the interface works in exactly the same way.

Appendix

The final code package for our application: FlashCards.zip

Presentation Slides: FinalPresentation.zip

Poster: Finalposter.zip

---