LoFi-Group:BluJay

From CS160 User Interfaces Fa06

Contents 1 Team Members and Roles 1.1 Bowen Li 1.2 David Hoffman 1.3 Jonathan Yen 1.4 Yimin Yao 1.5 Shared Roles 2 Introduction and Mission Statement (6 pts) 2.1 Backgrounds and Overview: 2.2 Mission Statement: 3 Prototype (12 pts) 3.1 LoFi Prototype Images 4 Method (12pts) 4.1 Participants 4.2 Environment 4.3 Methodology 4.3.1 Making Flash Cards 4.3.2 Importing 4.3.3 Moving Cards Between Folders 4.3.4 Study Mode 4.3.5 Changing the Comfort Level of a Flash card 4.3.6 Quiz Mode 5 Results (12 pts) 5.1 Making Flash Cards 5.2 Importing 5.3 Moving cards between folders 5.4 Studying 5.5 Quiz 5.6 Miscellaneous Interview Points 6 Discussion (12 pts) 7 Appendix (6 pts) 7.1 Bowen Notes RAW 7.2 Jonathan Notes 7.3 David Notes

Team Members and Roles

Bowen Li

• Interview facilitator / human computer
• Handling prototype materials
• Drafted consent form
• Developed software logic / interface

David Hoffman

• Interview observer
• Developed prototype
• Creating mock flash cards / look and feel of prototype
• Talking with flashcard users
• Developing solutions to how computers can meet the user's needs

Jonathan Yen

• Interview facilitator / observer / greeter
• Gained familiarity with target user group / found interviewees
• Has best understanding of what the users really want from the software

Yimin Yao

• Developed prototype
• Developed photoshop mock-ups
• Interview observer / human computer
• Finding interviewees

Shared Roles

• Develop prototype
• Come up with questions
• Note taking

Introduction and Mission Statement (6 pts)

Briefly introduce the system being evaluated. State the purpose and rationale behind the project as well as the experiment you are conducting. Then, present your mission statement. The mission statement should represent the common purpose and goal of the project. Each member of the team should agree on and be committed to achieving the mission statement. Describe the role of each team member for this assignment.

Backgrounds and Overview:

The BluJay Digital Flashcard system that we are currently developing is targeted towards students studying subjects in which flashcard making is frequently used as an effective tool for studying. Through our contextual inquiries in the previous part of the project, we have found many regular flash card users who recognize the difficulties in the efficient organization and studying of their flash cards; for example, the tediousness in randomizing and sorting of the cards to facilitate studying. Several users who make flashcards for more than one classes have also expressed their desire to better distinguish and organize their cards made for different subjects. Thus we have decided to design a digital flashcard system to provide flash card users with a system that helps them efficiently organize and study their flash cards, and perform quick assessment on their familiarity with the cards, without compromising the functionality of the original paper flash cards and users’ flashcard making habits.

Mission Statement:

Our mission is to provide a user friendly interface that bridges the paper flashcards and additional digital features using Anoto technology to help them study flashcards with maximum efficiency while minimizing the effort required from the users to adapt such system.

Prototype (12 pts)

Describe your prototype. Reference sketches of the interface screens in your description. Finally, submit a image file of the entire system with all of its elements laid out.

The Low-Fidelity Prototype used for experiments is developed using regular paper and pens. A typical screen of the prototype consists of four frames (fig.1a):

1. Conventional toolbar (on the top): File, Help, etc 2. Library (on the left): A column of directories for navigating through all the subjects of flashcards that have been entered into the system. 3. Main frame (biggest region on the screen extending to lower right corner): Region where flash cards are displayed individually for studying, by subject groups for browsing, or shown in a question format for quizzing. 4. “Mode” Tabs (above main frame): allow users to switch from one task to another (for example, from studying to taking a quiz)

For experimental purposes, the frames 1, 2 and 4 are primarily static and are presented to users as one entire sheet of paper interface; where as the main frame (3) is dynamic, replaced by different screens for each specific task (fig.1b).

figure 1a: Viewing cards in browse mode

figure 1b: Interacting with the study mode interface

figure 1c: Interacting with the quiz mode interface

The main frame (region 3) can display one of the following three different screens:

A. Browse (as shown in figure 1a):

• thumbnails display: all the cards in a folder. Cards can be selected to set comfort level, to be deleted, or to be moved to a different subject folder.

B. Study (as shown in figure 1b):

• single card display: a full size card is displayed for studying

C. Quiz (as shown in figure 1c):

• quiz home: set parameters for quiz including subject, level of difficulty, etc.
• quiz question: a multiple choice question that displays a front of the questioned card, with four backs (3 false backs plus the correct one).
• quiz result: display the correctness of each question answered and overall grade.

Other features of the prototype:

• Flashcard making:

Our low-fidelity prototype for making the cards are just regular paper in flashcard shape.

• Importing

As soon as the user inserts the pen into the “pen holder”, an import screen is displayed. The import screen prompts the user about the format in which they want to import the flashcard such as the way they flip cards.

LoFi Prototype Images

Importing screen: lets the user choose which folder to put the cards into, and allows the user to specify which direction the flashcards are flipped (this is important because of how the cards will be viewed digitally). The import screen will also let the user know how many cards are being imported.

Browsing screen: Will show the front of each card as a thumbnail, with a checkbox next to each card to indicate whether or not the card will be manipulated. It may be better to eliminate the checkbox and allow the user to click on the card so that it becomes highlighted, allowing for direct manipulation.

Study options screen: Allows the user to select a desired group of cards, and specify the manner in which they want to study the cards.

Study screen (sequential mode): Enables to user to navigate by clicking on the arrows under the card, or by clicking on a card in the "filmstrip" at the bottom of the screen. Also provides some options for studying and changing the rating for the current card.

Study screen ("smart random" mode): Same as sequential mode, except that the "filmstrip" is covered over. It is not clear what the additional space may be used for when it is covered up.

Quiz options screen: Gives the user a choice of cards to study, and generates a multiple-choice quiz based on the card selection.

Quiz screen: Shows the front of one card, then gives 4 choices for the user to pick from. After clicking on a card, the user may go on to the next card or quit the quiz.

Quiz summary: Shows the results of the quiz.

Some flash cards that were made by participants during the lo-fi prototype test.

Method (12pts)

Participants

• Participant 1: The first participant is a 4th year student majoring in Math who often uses flashcards to study GRE vocabulary and math equations. He binds his cards with key ring, but finds that studying cards in the same order reduces the effectiveness. Thus he would be interested in a program that allows him to study and quiz himself in randomized order without losing the original order of the paper flashcards.

• Participant 2: The second participant is a 1st year undeclared UC Berkeley student who used flashcards for SAT and is now making flashcards for his Korean course. He is chosen to participate in our study because unlike the other participants, he is less comfortable with using new computer software. He also brings us feedbacks from a new perspective as a younger user.

• Participant 3: The third participant is a 4th year business student. She was selected because she makes flashcards for both Chinese course and Accounting. As a typical college student whose life is heavily dependent on her laptop, she is very interested in such a digital flashcard system, and express the desire to test herself on the flashcards as well as organizing cards from the two different subjects.

Environment

All the three experiments were carried out in the fifth floor study lounge of Unit 2 dormitory. The place is chosen because that's where two of our subjects often study for classes, and it's a convenient location for the third subject as well. On the test day, the lounge was empty other than the project team and the participant. The lounge was fully equipped with clear large desk and comfortable chairs; high altitude and large windows provided excellent lighting. The participant sat across from the computer, with the facilitator on his or her side; the two note takers sat at two far ends of the desk to obtain a good view without disturbing the experiment.

Methodology

We didn’t have a “script,” but we did have the same procedure for each test user. Since each user behaved differently depending on their familiarity level with computers and flash cards, we decided it was best to react according to each user instead of having a set script. The procedure we followed is outlined below.

[edit]

Making Flash Cards

After explaining how Anoto and our program work, we had the user make a sample flash card for us. This accomplished one of two things: it got the user into the mode of flash cards. It allowed them to think about the steps they normally take when making and studying for flash cards. Second, they would have a physical flash card to look at (as they would when normally interacting with our program). This would help them with simple tasks such as deciding which way they flip the flash card, or deciding on how they want to study from the cards.

[edit]

Importing

Then we had the user “dock” the pen and import the flash card(s) they just created. The first screen from our interface that they reacted to was our “import” screen. On the import screen the user had to pick which folder to import the newly created cards to, and pick the orientation of the “flip” they use (vertical or horizontal flip). Here we assume a couple things. We assumed zero processing time from the raw pen data to our programs representation of flash cards. In the interview the users were able to immediately interact with complete “flash card” representations. We also assumed that the user would follow some sort of guideline on the physical flash cards to orient each flash card correctly. This is because without any guideline, the computer would not have any way of knowing which way is up/down, front/back for each and every card. With some guidelines, at least each batch of cards will have the same orientation, regardless of whether they are oriented “correctly” or not.

[edit]

Moving Cards Between Folders

After importing the flash cards, the program opens in the “browse” view with the list of cards the user imported open. Here we had the user move a subset of these cards into a different folder. This is done by checking the relevant cards, selecting the desired folder from a drop down menu and clicking “move.” Here we specify to the user to select the option to create a new folder.

[edit]

Study Mode

Once the cards are in the correct folder we asked the user to study from the cards in study mode. This is done by clicking on the “study” tab. The first screen is a prep screen that asks the user to select from a couple different options such as difficulty level of cards included and presentation order of the flash cards. After selecting these options and pressing “Go” the user is presented with a screen that shows flash cards one at a time. The user can then traverse the list of flash cards he or she selected in the options screen by using the forward or back arrows. The user can choose to view either side of the flash card or both sides of the flash card via radio button selections. Additionally we ask the user play with the “flip” button to flip the current card.

[edit]

Changing the Comfort Level of a Flash card

In study mode, we ask the user to play with the settings of each flash card such as changing the familiarity level. This is done by changing the radio button on the right. This can also be done in the “browse” view.

[edit]

Quiz Mode

The final task we had the users run through was the quiz function. The quiz function is accessed by clicking the quiz tab. The user is presented with a prep screen similar to the one they saw in the study mode. This time they do not get to select the order in which the cards appear. After hitting ok the user is presented with a “multiple choice” quiz. One of the card from the list is presented. The opposing side of 4 other cards in that list are shown as options and the user has to select the correct answer. When the user has traversed through all cards, a review screen is presented that shows the results of the test. There is a summary of number correct and a listing of each individual problem. This section of our program was most open to change and we welcomed user feedback about what functionality they would like to see in the quiz and quiz results.

Results (12 pts)

Summarize the results of the experiment from your process data.

All three participants were able to maneuver through our paper interface and complete the tasks that we specify for them with minimum amount of difficulties due to their familiarity with the conventional interfaces. Each participant encountered some cases of minor confusion or performed steps unnecessary for the tasks at some point during the experiment. The following is a brief summary of problems encountered during each task completed by the participants.

Making Flash Cards

This step is relatively trivial since we don’t have the real Anoto paper setup. Two of the three participants reached the back of the flashcards by flipping the flashcards horizontally, while the other user flipped the card vertically.

Importing

The original design was to have users import their new flashcards into a default unsorted folder if the desired subject folder has not been created, and then transfer these cards into desired folders later. However, when the task of importing is given to Participant 1, he had trouble selecting the folder to import his new cards because he could not tell what “unsorted” folder was for. Thus after group discussion, we decided to modify the interface to add a “new folder” option where user can enter the name of the subject folder these flashcards belong.

Both Participant 2 and 3 were able to successfully import the data after the addition of “new folder” option. Participant 3 was temporarily confused by the diagram illustrating the flipping orientations; but we conclude after discussion with her that the confusion was caused by the inaccuracy of the low-fidelity sketch.

Moving cards between folders

All three participants immediately perceived that they were supposed to check the cards they intended to move, and had no trouble locating the "move" panel at the lower left corner of the main frame.

After moving the cards, participant 1 and 3 thought that selecting the comfort level was the next step, and they tried to set the comfort level of the cards. However, because they had just moved the cards to another folder, they couldn't set the comfort level! We postulate that part of the reason is that the interface design grouped all the buttons that acted on the cards towards the bottom. And in using most of the buttons for the "move" action, the users probably grouped the "set comfort level" button as another step in the same task, when in reality it is a separate task. Another possible reason for this confusion is the lack of immediate feedback. On the paper prototype, clicking "move" doesn't actually move anything. So if the cards were actually shown to be moved to another folder, and they no longer showed up on the current folder, the user may realize that they cannot set the level of non-existent cards.

Studying

Participant 1 started by clicking the subject folder he was trying to study. Realizing there’s no study option in the main frame where all the cards in the folder were displayed, he scanned through the rest of the interface and found the “study” tab on top of the main frame. At the study main page, he selected the desired folder and checked "unfamiliar" flashcards. He was unsure what the term "Smart Random" meant in the traversal options. Also, he noted that he thought it was cool that we had the feature to "show less familiar cards more often"

In the study screen, the user focused greatly on the center of the screen, pressing the forward and back arrows very frequently. He did not press the "go to end" arrows at all. The user wanted to study from the back side of the cards instead of the front side of the cards, but didn't see the option located on the right side of the screen. When asked why he didn't notice that option, the user said he was too focused on the center of the screen. Additionally, he stated that his first instinct was to ask the team for help instead of trying to figure out where the functionality was located.

Participant 2 behaved slightly differently from participant 1. User 2 clicked the study tab first without going clicking on the folder he wanted. There he was able to select the correct folder. However, when choosing the type of cards to include, he didn't realize that the check boxes were check boxes. He thought they were radio buttons, which has a very different behavior for inclusion of objects. We are not sure why he didn't recognize the check boxes in this step, when he correctly identified them in the "browse" mode. This user chose to study from the flash cards by viewing both sides of the card at once.

To get to study mode, participant 3 also clicked on the subject folder in the navigation frame first. However, since the team noticed this behavior from the first user, we changed the behavior of the "study options" screen to select the folder the user came from by default. Thus for participant 3, the extra click to the right folder in the navigation pane had some useful effect instead of none. Since the user had previously decided to make all of her flash cards the same difficulty, she could only select one difficulty setting and the "show cards I know less well more often" option was not useful. She was able to complete the other tasks without trouble.

An interesting note is that the users pressed the "go to end" forward or back arrows VERY little (if at all). They mostly used the forward and back arrows. Additionally all the users had no trouble understanding or using the "comfort level" radio buttons on the right hand side.

Quiz

All three participants were all able to locate the “quiz” tab and click it as their first step because they have learnt the behavior from the previous study task. All three also had no trouble working through the "quiz options" screen because of its similar look to the "study options" screen.

Participant 1 thought it was interesting that we had a "quiz" function with multiple choice answers. However, he felt that in real usage, he may be able to "cheat" the system by process of elimination. Choosing doesn't force the user to know the right answer, only know which ones aren't right. Participant 1 selected the answer by clicking on the card he wanted, and then hitting the ok button. At this point in time, it wasn't clear what the response should be from the program perspective. Should the correct answer be displayed, or should the program move to the next program? User 1 preferred seeing the answer right away. The team decided that it was a good idea to put this as a selection in the "quiz options" screen for future users. When the user finished the quiz, he was shown the "results" screen. On this screen the user said he would like to be able to click on individual questions and go back to the original question. He said that having the answer "in-line" would become too cluttered. He also expressed concern for the list becoming too long if the study list was very long to begin with, and suggested pagination of results.

Participant 2 also selected the answer by clicking on the card and clicking 'ok'. Participant 2 chose to get immediate feedback about whether he was right or wrong. User 2 displayed an interesting behavior at the last page of the quiz. He pressed the "quit" button when we informed him it was the last page. This quits out of quiz mode, instead of taking the user to the results screen. Thus the team decided that the wording on the screens should change when it is the last screen: the "ok" or "next" button should be replaced by "finish". Once on the results screen, participant 2 also said he would like to be able to click back to the original question. However, he equally liked the idea of in-line answers on the results screen to save him from clicking.

Participant 3 was able to traverse through the quiz function with no trouble. She decided to "wait until the end" to see her results. On the quiz results screen, this user said she did not like the "in-line" option of viewing the answer, but instead would prefer to click back to the original screen. She said this is because sometimes the context of the original question helps her remember better.

The team left the "done" button at the end of the quiz open to the user. Participant 1 postulated that it may take him back to "home" or to the "browse" screen. Participant 2 wanted the button to go back to the beginning of the quiz mode. Participant 3 said the button should go back to the "browse" screen.

Miscellaneous Interview Points

Out of the three users, only participant 1 found that if the tabs the top changed without him clicking on it directly to be confusing. The other users said they would have no problem with this behavior.

In the browse mode, it wasn't clear to 2 of the users what "rotate 180 degrees" meant. We assume that this is because in our lo fi example, we did not have any upside down cards as our examples.

User 1 said that he likes to play around with programs and click randomly to see what different options do.

User 2 said that he would like to have the flash card he made last show up first in browse mode, because he usually sees the first card a lot when studying. However, he noted that if studying on the computer is random rather than linear, this isn't as big of an issue.

All users we talked to referred to the "front" of the flash card as having the word or name on it, while the definition is considered to be on the "back" of the flash card.

User 1 wants the ability to print flash cards from the library.

User 3 hopes to be able to create folders within folders, because sometimes she has multiple topics within a class that she would like to be able to separate.

Discussion (12 pts)

Discuss your results. What did you learn from the experiment? How will the results change the design of your interface? Was there anything that the experiment could not reveal?

One of the most valuable pieces of information that we gained from our lo-fi prototyping is how users react to the flow of our program. As designers we create one or more ways for users to get tasks done. They have to carry through certain steps in a certain order to get the task done. What we want our program to do is present the steps in such a way that matches the user’s conceptual model of those steps: they should match. The differences were manifested through user error.

This occurred first when users were choosing to move card to a different folder. Most of them believed that selecting the “comfort level” was part of this task. However we had not meant it to be. Since many of them were following the buttons from left to right, we think that moving the comfort level button to another location may solve this issue. Perhaps moving it to the leftmost side of the bottom bar will solve this problem, as the ‘move’ task tends to be left to right oriented, it will no longer interfere with the “comfort level” button. Also, another option is putting a divider line between the “move” and “comfort level” tasks. There is currently no divider.

One user said that using selection menus was not satisfying because it doesn’t feel like anything has been selected. Our program is heavily dependent on selection menus for selecting different folders, so it is important that this aspect of the program work smoothly. For our next round of prototyping, we will explore alternatives to using a drop down selection menu.

One area of major feedback is the quiz “results” screen. It seems that different users have different preferences as to the method of feedback they receive. However, ALL the users specified the desire to review their performance in detail after the quiz. This point is important because it means that the focus of the quiz functionality is not only to quiz the user, but it is to quiz and review the user. Thus the team will definitely incorporate more reviewing functionality (such as clicking back to the original question) in the next prototype.

Although there was much discussion about the logistics of when to click “next” or “ok” on the quiz itself, this aspect of the program may be considered as a secondary concern. This is because although it may not be immediately clear what the behavior will be if the button is pressed, there is very little margin for error for the user. Unlike the example of moving the card and then selecting the rating option (which is a logical error), highlighting and clicking ok, or just clicking the answer does not produce an error either way. The user may be slowed down, but won’t be stopped from performing the task.

Another area that showed a flaw in our system design was in study mode. If the user changes the comfort level of a particular flash card during study mode, the response from the program is undefined. This is because in the study 'options' screen the user selects which cards to study from by comfort level. For example, the user can select to study from only 'unfamiliar' words. Thus if the user is in study mode and decides a word has become 'familiar' they can change the comfort level on the right. But the uncertainty comes from whether the new 'familiar' word should show up still in the study list. The word was part of the original list of 'unfamiliar' words but is no longer 'familiar'. The group is currently undecided as to which is the desired behavior. One reason to take the word out of the list immediately is if a user changes a comfort level then they are signifying that they don't want to see a word they already know. However, this may break future analogies of coming into the study mode with a set list of words and going through that list.

There were also some minor issues the team ran into while conducting the interviews:

• the need for a “default” folder
• when viewing both sides, the “flip” button is not needed
• would be nice to see more options in browse mode (rating of cards, able to selectively view by rating, etc.)
• “smart random” doesn’t mean anything: use more useful terminology

One of the weaknesses of the lo fi interview is that it did not provide us with feedback on what elements the program lacks. Because the users start with being presented with an interface, they are “stuck” in that frame of mind. It’s hard to them to come up with applications that they may want or need because they don’t see it in front of them.

Appendix (6 pts)

The appendix should include copies of all materials involved in the experiment. This includes your consent form, demo script, and any instructions or task descriptions you handed out or read out loud to your participants. Finally, it should include all the raw process data you gathered during the experiment. Merge the critical incidents logged by the observers and list them here. The appendix materials and screenshots do not count in your 6-10 page total.

Bowen Notes RAW

Problems:

• • What does the "done" button do?
• • Changing tabs is confusing
• • To go to study, people tend to click on the folder on the side first, then the study tab (O->V). But the study tab makes them do the selection process over again
  • o What does clicking on the folders to the left do?
  • o Are they useful?
• • Our drawing of "which way do you flip" is confusing
  • o Users have to make the flash cards in the correct orientation as per our program
• • Using a selection menu doesn't "feel" like you selected anything (more feedback?)
• • Not obvious what "flip" does, or if it's permanent
• • What does "go to end" buttons do on random?
• • If user changes a rating in the middle of "study" does that word disappear from the list?
• • People flow using the bottom buttons: select all, select folder, move, set level
  • o The eye follows movement, but you can't select after the cards are gone
• • When is it appropriate to click "ok" / "next" in the quiz mode? Does clicking on an answer select it or select and confirm that answer? Does the user get more than one chance?
• • Does leaving quiz mode not through "quit" save the data?

Feature Requests:

• • Ability to print out flash cards
• • Want to be able to create subgroups (our example led user to believe you would only create class folders)

Feature requests (internal):

• • Browse mode: show # of card and comfort level
• • When "view both sides" selected, the "flip" button is invalid
• • Need "unsorted" folder -> though this could be better worded: what does it mean?
• • "full view" mode
• • Browse mode: "show only difficult, etc"

User1:

• • User flips horizontally
• • Note: need new folder option for where to import to
• • What does "smart random" mean?
• • When someone clicks between tabs, we need to stay within the folder
• • "when wrong, adjust"
• • No "back" button / functionality in some places
• • Front: word; Back: definition
• • Blank = front; Lined = back
• • He likes to use lined flash cards for vocab, blank ones for math
• • What does the "flip 180 deg" do?
• • Likes to play around and press buttons to see what they do: -> need back functionality

Study Tab:

• • Both forward AND back buttons are very important
• • User was focused on the very center. He didn't see the options on the side

Quiz Tab:

• • Thought it was cool that we had the option of "show difficult more often"
• • Have functionality of "check answer right away" or "don't see answer"
• • On the last question, don't have a "next", but instead have a "finish" button
• • Would like question #s in the results link back to the original question page
  • o Or show it on the screen; but this may be cluttered
• • Would like to have pagination if there are excessive questions
  • o Or possibly some "click" and have it expand within the page itself (web2.0)
• • Would like to be able to "select all incorrect" to highlight or switch back to browse tabs
• • "Done" would go to browse or a "home" page
• • The multiple choices: he can know this by elimination
  • o "smart" choices for the possible answers would be ideal
  • o Or maybe something more open-ended

User 2:

• • Whichever card you wrote last, make that show up first in "browse" because you see the first card the most: doesn't really apply if you use "random"
• • See immediate results
• • Had problem recognizing a check box in the "study" and "quiz" intro screen

User 3:

• • Front: word; Back: definition
• • Prefers to click back on quiz results instead of inline because sometimes the context of the question helps you remember
• • Tab switching is ok
• • Wait until the end to see results
• • Made all her cards "familiar" setting -> why?

General Notes:

• • All of the users followed the "select" check box option pretty well
• • What if you have too many folders -> scrolling on the import screen, or study, or quiz screens becomes a hassle
• • Change "import" / "cancel" to - "import" / "leave data on pen"
• • What happens if you open the program w/o import -> what is the first screen?
• • In "show difficult cards more" - is it mostly random or mostly sequential?
• • Maybe have a "title" for "study" sequential - so it's obvious that the settings apply for that card specifically

Jonathan Notes

User 1:

• Import task: user needs a way to make a new folder
• Tabs: what happens if the user needs to change to a different mode in the middle of a task?

Study mode:

• Smart random vs. random: unclear what the meaning of smart random is
• No “back button” after selecting preferences for study mode
• “Show both” option in study mode: what should be done about the “flip” button

Quiz mode:

• Next/Quit buttons: will this be necessary? Other possible ways to implement (i.e., just go to next card after clicking on an answer, can quit by just switching to a different mode or exiting the program)
• Immediate feedback: can have a “check answer” button rather than having immediate feedback; could also just give feedback at the end of the quiz
• Should show problem number (i.e. Problem #3 of 10) in the corner
• Quiz summary: would be helpful if more detailed feedback provided (possibly have this as a separate mode)
• Quiz summary: “select all incorrect” feature

User 2: Study mode:

• “Study from” option: should this change back to a default option after going to another card?

Quiz mode:

• Need to have a preference for feedback before beginning the quiz
• What does the quit button do: will this move the user back to the home screen, make the quiz idle, etc?

Other issues:

• Flip choice in import task: not obvious what the options are, and might be irrelevant depending on how the paper interface is implemented
• Need to swap “set comfort level” and “move/copy” in the browse mode
• Possible situations in which a user may want to browse multiple folders at the same time?
• Flash cards in the GUI should be more rectangular

David Notes

Raw Lo-Fi Data

User A:

• Observer 1 notes:
  • Subject writes a math equation on one side of card, answer on reverse side
  • Confusion over the number 128 printed in LoFi import window, (told that this is the number of flash cards he would have just imported)
  • Requests a new folder option because none of the preset paper ones match what he wanted
  • Doesn't understand what the flip 180 degrees does, but says he would just play around with it if it were a real computer until he figured it out
  • Unsure what Smart Random presentation order does (in study mode, this makes sure that all the cards are shown i.e. random order, not random presentations
  • Didn't notice the radio button where you can specify what you want to see first.(Understood it after having his attention directed to the controls)
  • On the Quiz section, he wanted immediate feedback, did not want to wait until the end of the quiz.
    • Suggested possible interface where there is a check answer button during quiz, where the first response is counted for the overall score but the correct answer is given before moving on to the next question
    • On results page wants to have hyperlinks to the questions
    • Alternative, is to select all incorrect and review them in study mode
  • expected hitting the done button to take him back to quiz setup, not to a different mode such as browse window.
  • In final thoughts:
    • • wanted a print flashcard mode to take advantage of program organizational capabilities
      • Thinks 4 choice Multiple choice quiz will be too easy
      • Thinks that flashcards should be blank on one side and have lines on the back (Sample flash card was square graph paper on both sides)

UserB:

• Observer 1:
  • Chose to use the select all during import
    • Clearly only makes flashcards for one class at a time
  • Wants to be able to use technology in library(so software needs to work on laptop)
  • When studying, like the show both sides feature
    • This is a feature that would only be available on computer, real flashcards have a boolean output(front or back, but not both)
  • Thinks that there should be defaults on the quiz preference
    • Default should be to test everything
    • Didn't want to enter the test information each time
  • Wants a mode on the quiz mode where you can select show answer or wait til end
  • add a go back button

UserC:

• Observer 1:
  • found the card flipping symbols confusing(claimed she could figure it out but the diagrams were not clear)
  • Possibly misunderstood the comfort level settings
    • Asked the program to test her on the words she knew
    • Didn't speak enough to let us know what she thought she was doing
  • She wanted to organize the cards into more detailed categories
    • group by class is not good enough
    • suggests adding nasted directory structure so it is groupable by midterm or even topic
  • Especially appreciated the random study feature
    • in regular flash card usage, says she always learned the first card best
  • On quiz mode felt it was best to not see the answers until the very end
  • At end commented that the program is really cool and would be interested in using if it were fully developed.

---