LoFi-Group:Happy Toast Mobile

From Cs160-sp08

Contents 1 Group Members 2 Introduction 3 Mission Statement 4 Prototype 5 Method 5.1 Participants 5.2 Environment 5.3 Procedure 5.4 Tasks 5.5 Test Measures 6 Results 6.1 Winifred 6.2 Dirk 6.3 Bilbo 7 Discussion 8 Appendix 8.1 Lo-Fi Prototype Images 8.2 Task Descriptions 8.3 Demonstration Script 8.4 Informed Consent Form 8.5 Critical Incidents

Group Members

David Jacobs - Helped design and construct the paper prototype; Acted as the computer during user testing; Outlined and tweaked several sections of the writeup; Assistant to the prototype photographer :).

Tam La - Helped design and construct the paper prototype, and was part of the Prototype and the Discussion section. Also took the prototype photos.

Lita Cho - Help design and create the paper prototype. Helped write the Prototype and the Discussion sections. Also wrote the script for the study and acted as the facilitator.

Ben Sussman - Acted as an observer during user testing. Drafted the introduction, and methods section and authored the Mission Statement and the Informed Consent Form as well.

Roseanne Wincek - Acted as an observer during user testing. Compiled notes and critical incidences. Wrote methods, results, and discussion sections.

Introduction

BorgTV is an Android application designed to aide users in the remote control of complex multi-component media centers and to alleviate some of the difficulties associated with configuring and using universal remote controls. There are numerous problems with existing universal remote control solutions, including user difficulty regarding the modes of the device, finding and setting up the remote with numerous devices from different manufacturers, and simply confusing the user with too many buttons. Borg TV will be specifically designed to solve these problems and also be a part of the user's phone so as to be more difficult to use and prevent the "device clutter" which often plagues many living rooms.

Mission Statement

With BorgTV, we intend to provide a simple, clean, and fun-to-use interface for remote television and media center control. This low fidelity prototype reflects the habits, difficulties, and frustrations our group observed while conducting our contextual inquiry interviews. We wish to apply these observations to create a more straightforward and accessible interface for users universal remote controls.

Prototype

Before we could construct a prototype, we had to rough out a new, touch-screen-less interface design. The primary challenge was allowing the user quick access to any number of devices and functions without resorting to arbitrarily (and likely invisibly) mapped hot keys -- the guide interface mapped easily, and is not discussed in detail here. In order to keep our interface visible, we decided to constrain our design so that all functions could be accessed using solely the D-Pad. Initially we started sketching interface ideas on white boards, scratch paper, lecture notes, etc. One suggestion involved a 4x3 grid of on-screen buttons where each button represented a location/room/device/function depending on where the user was in the device hierarchy. However, selecting the desired button from that many options using the D-Pad alone proved inefficient in pilot tests.

Eventually, we settled on a 3x3 grid with two button types. The middle five buttons forming a "cross" provide choices one level deeper in the hierarchy (i.e. in a room screen, the five buttons would be device names). The four corner soft-buttons remain fixed throughout the hierarchy for consistency. We decided that the lower right button should be a "More" option, showing a list of options and other locations/rooms/devices that weren't able to fit within the middle five buttons. We decided that the lower left button should be a back button that goes back up one level in the hierarchy. The menu and the back hard-buttons provide the same functionality as the lower right and lower left soft-buttons. However, the team decided that redundancy was a positive trait because the user can have many means of performing the same action, and it increases visibility. The team thought that the user should always have access to the TVGuide, since this is a primary feature in our application, so we made the upper left on-screen button to be a fixed Guide button throughout the hierarchy. Lastly, we decided that a Help button should be fixed on the upper right, so the user would always have access to some sort of documentation, no matter where they are within the interface. The resulting help dialog would be, of course, context-sensitive.

The 9-button screen works well for hierarchy navigation, but requires at least two button presses to select an option. For device operation, this could quickly become bothersome, so we designed a separate, one-touch interface for the leaves of the hierarchy. Function sets, as the hierarchy leaves are called, appear as a D-Pad with functions overlaid on each direction and the center button. This indicates that the user can press the D-Pad button and the action described on the mapping will happen with a single button press. Uncommon device functions (like closed captioning) and those that don't fit well in logical groups of 5 buttons (such as the number buttons) can be accessed via the the More menu.

After redesigning the interface, we created a state transition diagram defining the computer's behavior -- it is not attached here for the sake of the reader's eyesight. Additionally, with all the screens designed, we were ready to create a catalog of requisite UI elements that needed to be mapped into a paper implementation for the low fidelity prototype. The list of elements and their paper incarnations follows:

Screens

To emulate the phone screen on paper, we mounted printed paper rectangles (3"x2.25" -- the correct aspect ratio) on half index cards (3"x2.5") so that the screens were sturdy and the computer would have a easier time handling and switching screens. We used the 0.25" border to label to the various screens and cue the computer on the ordering of the screens, depending on the task. The screen was numbered by task, and the screen number it appeared as within that task. The flash cards were color coded as well. Orange cards indicated a 9-button-screen, green cards indicated a device D-Pad, red cards indicated a dialog box, and yellow cards correspond to the guide interface. We made generic templates in Adobe Illustrator for common screens, such as blank 9-button-screens, D-pad mapping and dialog boxes. However, all the titles, button names, and various details were written or drawn by hand.

Hardware Phone

To simulate the Android cell phone, we printed a large-scale image of the QVGA-L phone shown in figure (See Figure 1, all figures are in the Appendix). We mounted that image onto stiff poster board for easier handling and better durability. The team also created multiple copies of the phone for screens that require extra set up, such as scrolling lists (see below), in order to expedite the study with the user.

Buttons

All buttons throughout the interface are represented as identical rounded rectangles. They are all the same size in order to maintain consistency and help ease button focus management during the prototype testing (only one acetate translucent highlighted region is needed for all soft buttons). The 9-button screen template emphasizes the two button groups by drawing buttons over a grey cross shape (See Figure 2).

Cursor Focus

Large pieces of acetate with translucently highlighted regions were used to act as a focus cursor (See Figure 2). After the user pressed the D-pad, the computer would move acetate in the direction of the pressed button to highlight the newly selected soft button displayed on the screen. This could be an up, down, left or right button on the D-pad. The middle button is designed for selection during navigation. Different acetate sheets were made for different highlighted elements (e.g. soft buttons, text fields, list elements).

Vertically Scrolling Lists

Simulating a vertically scrolling list with paper and markers was the greatest challenge we faced. How was the computer to emulate a fluid scrolling as the user goes through items on the list one by one? Since we wanted our lo-fi prototype to simulate the real program, we came up with a mechanism to emulate that action: We created a special paper phone for specifically for scrolling lists. It had slits cut along the top and bottom of the screen, through which long lists (about 4 screens in height) could be threaded (See Figure 3). The user could only see the on-screen options at any given time (See Figure 4), and the computer controlled the scrolling by pulling the list through the slits.

2D Scrolling Grids

The TV guide required navigating a 2-dimensional grid of times and channels. We thought about cutting a hole in the hardware phone and using it like a "peephole" display to make a 2D scrolling grid, but the mechanics didn't work well. Specifically, we needed certain elements on the screen to not move. For example, the vertical channel number listing and the horizontal time listing needed to remain fixed when moving in the other dimension. Ultimately, we chose to represent the BorgTV guide as two vertical lists, one for each hour in time. We made an overlay screen with the time hand written on the top. When a user scrolled horizontally we just switch the list to the next time frame list. As a result, we were able to simulate the horizontal scroll. Vertical scrolling worked just as in normal lists.

Radio Buttons

In the new device setup screen (See Figure 5), users select their device using a set of radio buttons. Each of the buttons is drawn by hand with a small circle adjacent to each option. To show the currently selected device, we made an orange acetate dot to signal selection (cursor focus used green highlights). Selection can be made using the middle button on the D-Pad.

Text Fields

To represent a text field we drew long rectangles on the screen templates (See Figure 5). As typical users are already quite familiar with QWERTY keyboard operation, we allowed users to "type" in text fields by voice. Text fields are filled using pre-written paper slips containing expected user input.

Method

Participants

This study involved three participants: Winifred, Dirk, and Bilbo. Winifred is a postdoctoral fellow at the QB3 Institute on campus. She has a moderate amount of experience with technology and media centers and is in her mid-30's. Dirk is a 4th year undergraduate bioengineering student with a significant amount of technological savvy and media center use. Bilbo is a 2nd year undergraduate MCB student who, while having a moderate level of experience with high-technology devices, rarely uses media centers. Efforts were taken to find a diverse test group, with different backgrounds, usage levels, and amount of tech acuity. Furthermore, Happy Toast strove to find test users with little personal connection to the group members themselves.

Environment

The user interviews were conducted in one of the group member's television rooms. This room was well suited for the task at hand, with multiple couches, a large wall-mounted television, and coffee table. The user sat on a couch, facing the television with the prototype on the coffee table in front of him. The team member playing the roll of computer sat opposite the user, just below the television. This allowed the computer to change the prototype screens as needed as well as manually control the television, providing realistic feedback for the user. The facilitator sat next to the user on the couch, with full access of the prototype during the demonstration portion of the interview. The interviews were video recorded by a camera placed on the table. This camera was configured in such a way that only the prototype itself and the hands of the computer and user are visible. All users signed waivers agreeing to participate in the study and allow video taping prior to the start of the interview (see appendix for consent forms). The room was large enough and with ample seating to allow all five Happy Toast group members to watch the interview. The three interviews were conducted sequentially, with each user being tested independently (waiting participants were seated in a separate room). The overall session lasted roughly two and a half hours; each user spent about 30 minutes interacting with the prototype, then was asked to give general comments during the debriefing.

Procedure

Tam La acted as the greeter for these interviews. His role was to show participants to the location of the experiment as well as sit with participants while others were being interviewed. Tam also prepared and broke down the room for the interviews. David Jacobs acted as the computer. He moved the cursor for the user, switched between different screens, and manually controlled the television. Lita Cho acted as the facilitator. She explained the Android platform, the BorgTV application, and the role of this low fidelity test to the user at the beginning of the session. She then performed a scripted demonstration for each user before asking them to complete his or her tasks. She also lead the debriefing questions after the tasks were completed. Roseanne Wincek and Benjamin Sussman played the role of observers, taking notes during all interviews.

Lita began all interviews by giving a brief explanation of the role and purpose of this experiment. She then explained the low-fidelity prototyping process and the importance of the user "thinking aloud" during the interviews. Lita demonstrated how the user that he was to press the buttons on the phone print out as well as verbalize the button being pressed so that the computer could interact accordingly. The prototype started at the home Android screen, and Lita began her demonstration by choosing the BorgTV application from that screen. Lita's demonstration task was to choose the television in her home bedroom, turn it on, and mute it (see appendix for demonstration script and task materials). This task was chosen as the demonstration task because it was relatively simple, yet showed the basic hierarchy and navigation in our system. After this demonstration, Lita asked the user to perform three tasks in increasing difficulty, beginning from the D-Pad screen where she left off.

Tasks

Each user was asked to perform a series of written tasks (see appendix for task materials). The task descriptions provided the required information to complete the task, but did not suggest a means of doing so.

The easy task that was asked of the user was to move to his home's living room and change the channel on the cable box to 48. From the bedroom TV's power-ops D-Pad, the user had to navigate the device hierarchy to the living room and find the cable box device. He then had to discover how to change channels. Viable options included changing channels one at a time with the CH/VOL function set, selecting 4 then 8 from the More menu, or simply pressing 4 and 8 on the QWERTY keypad. The user was not told how he should complete the task, nor did he know which channel changing functionalities were available.

The second task the user completed involved utilizing BorgTV's TV guide functionality. The user was told to find the time for Lost on ABC later that evening and to then set a reminder for it. From the ending point of the previous task (which varied depending on the user's choice of channel changing method), the user had to find the BorgTV guide. Once he had done so, the user entered the guide interface such that neither the correct channel nor time for Lost were on-screen initially. The user had to scroll vertically to find the appropriate channel, and then horizontally to find the proper time. Once found, the user had to select the television show. Doing so took the user to another screen with a synopsis and show information as well as icons on the bottom. The user had to select the bell icon to set a reminder. The user was not told that the reminder functionality was indicated by a bell.

The most difficult task asked of the users was to configure a new TV at their friend Bob's apartment. In order to do this, the user had to first find the root of the device hierarchy to create the new locations. Creating a new location could be done either through the More menu or by selecting one of the unassigned locations in the 9-button screen. Configuring a new location requires entering a location name and cable service provider in on-screen text fields. The user then had to configure a new room in Bob's apartment, the living room, so new devices could be grouped. This required simple text field entry as well. Finally, the user had to set up the correct IR codes so the phone could control the TV. This involved choosing the type of device (via radio buttons), television, and the brand (via text field). The user then went through the procedure for testing IR codes for the device (a series of dialogs with buttons for user feedback). Once the proper code was found, the user was asked to change the channel on the TV.

After all tasks were completed the user was debriefed on his experience with the interface. The user was asked to indicate which tasks were intuitive as well as which tasks were confusing. Team members asked the user about his decision making process through the various tasks. The users also were asked for suggestions about how unclear areas of the interface could be improved.

Test Measures

During the experiment we were looking for the reasoning and/or mental models that the participants had while they interacted with the application. This was primarily the task of the observers who noted any details which would inform us as to the logic behind their decisions. Our test was designed to measure the speed at which the user became familiar with the interface as well as the overall speed that the users were able to navigate through the interface. Accuracy was an easier detail to measure as mistakes were more obvious. We focused closely on any mistakes the participants made and then, during the debriefing after all tasks were completed, asked the participants pointed questions about their reasoning during the time of their mistakes. These questions helped pinpoint errors in our understanding of the user’s conceptual model as well as give us (the designers of the UI) knowledge about how users will react to confusing or under-explained elements of interfaces.

Results

Winifred

Winifred was the first subject to use the prototype. She completed the first task very quickly, without hesitation or confusion. However, at the end of the first task, she pressed the home key to start the new task. Pressing the home key took the user directly back to the Android home screen. The user was expecting to return to the BorgTV home screen. This expectation was common with all three users, and Winifred pressed the home button after each task. She expressed frustration that the home button did not return her to where she expected. For this user, this incident was rated a 4, due to the level of confusion experienced by the user. The user changed the channel using the up/down functionality mapped to the D-Pad. The majority of time spent on this task by Winifred was actually scrolling through channels with the D-Pad. At first, she tried to use the screen of the prototype as a touch screen and expressed her desire to have touch screen functionality. This incident was rated a 3. However, due to the limitations of the project, touch screen capability is not available.

Winifred also quickly mastered the second task, which involved the TV guide. She had no problem finding the guide button on the screen. When looking for a television program in the future, it was not immediately obvious to Winifred that she needed to scroll through the listings. However, as soon as she realized the layout of the program listings, she found the desired program with ease. This initial confusion was classified as a critical incident with a rating of 2. Once the user selected the appropriate show, she instantly recognized the bell icon as indicating a reminder. She easily scheduled the reminder.

The most problematic task for Winifred was the most difficult task of setting up a new device in a completely new location and room. She quickly recognized "New..." to initialize the set-up procedure. Her greatest difficulty with this task was setting up the room in the new location. She went back and forth between the room and location screens twice before realizing that she had to initialize a room in the location before she could initialize the device. The user found this especially frustrating. She used the more menu to configure the room in the location rather than setting it up directly from the location screen. This incident was rated a 5. It was not immediately obvious to Winifred that she still had to set up a device now that she had set up the room. However, she quickly reasoned that device set up was the next logical step. This incident was rated a 3. Finding the correct IR code for the television was extremely confusing to the user, and she had to consult the help dialog to figure out why the codes kept failing. When she did find the proper code, she was unsure if it was indeed the correct code. In the debriefing, she identified the lack of feedback as the cause of her confusion. This incident was rated a 5.

Dirk

Dirk quickly mastered the interface. He employed the phone's back button, rather than the home button, to return to higher levels within the interface hierarchy. Dirk chose to use the phones number buttons to change the channel instead of scrolling through channels using the D-Pad. He said that he surmised that functionality by the enter key mapped onto the D-Pad. Dirk had no critical incidents while performing this task.

In order to use the TV guide, Dirk found it through the menu button on the phone. The menu list was very long, and Dirk expressed frustration that there was no page down functionality. This incident was rated a 2. The TV guide is, in fact, not listed in the menu. After Dirk had scrolled through all options, he still had not found the TV guide. This incident was rated a 3. Dirk quickly found the TV guide as soon as he returned to the room screen. He easily navigated the guide listings. He also easily identified the bell icon as the way to set reminders. He also correctly identified the three bell icon for setting recurring reminders.

To begin the third task, Dirk tried to use the home button to reach the home screen of the application. Dirk also cited this issue in his debriefing session. This incident, like Winifred's was rated 4. The user was unsure of how to being setting up the new location and had to consult the help screen. He then tried to use the "More.." button, but could not find the option he was looking for. Dirk was not in the correct place in the application's location hierarchy to configure a new location. This incident was rated a 4. Once he started the configuration procedure, he had no problems. He quickly understood the method for finding the proper IR code. In the debriefing, he thought that the user had to go through too many menus and screens to perform simple TV tasks. This issue was rated a 4.

Bilbo

Bilbo's interview began with his use of the back button to navigate through the location hierarchy. He quickly found the proper function set and scrolled through the channels using the D-Pad.

For the second task, Bilbo easily found the guide button on the room screen. He quickly navigated through the channel listings without issues. Although he was unsure of exactly how to find the show in question, he intuitively scrolled through the listings and promptly found his desired show. He expressed that he was not certain how to set a reminder, but thought aloud that he should select the show in question to see which options were available to him. He effortlessly identified the bell icon as the reminder indicator and set a reminder without incident. He said that he was unsure of the meaning of three bells. This incident was rated a 1. When he arrived at the reminder conformation dialog, he commented that he would choose "OK" since it was the only option available to him. This incident was rated a 1.

Bilbo was confused about his location in the application hierarchy when beginning the third task. Although he recognized that the application was set to his own home and living room, he tried to configure a new device. It was not immediately clear to Bilbo that he had to set up a new location in order to control devices at another home. He was able to configure the television even though the location was not correct. This incident was rated 4. He was also unsure of the brand of his friend's television, and tried to set up the television without a brand. He got to a dead end screen instructing him to enter the brand name of the device, that was written on the television before him. The incident of trying to configure a television without the brand was rated a 2. When testing the IR codes, Bilbo tried to use the screen as a touch screen and expressed desire for such functionality. Bilbo easily found the correct IR code for the television, although the device was demarcated as TV2 since the new location was never configured. After setting up this TV, he realized that the location was set at his apartment, and tried to find Bob's apartment in the hierarchy. He then went through the motions of setting up a new location, and then had to set up the same device again. This incident was rated a 4.

In the debriefing, Bilbo expressed his confusion associated with the home button. He also expected the home button to take him to the application home screen instead of the Android home screen. This incident was rated 4.

Discussion

In this experiment, the tested users could complete all tasks asked of them. It was found that, in general, the interface was relatively easy to use and all three tasks took no longer than 25 minutes for each subject to complete. Even when the user was unsure of what choices to make, generally the user's intuitive guesses were correct. However, there were areas of the interface with which all tested users struggled. Major usability problems were encountered that must be addressed in future iterations of the BorgTV interface. The most critical issues users had with the interface were the lack of a touch screen, confusing mapping of the home key, and issues with usability and understanding of location/room/device hierarchy.

From this experiment, it was obvious that these tested users wanted a touch screen phone. When asked to perform tasks, all three users instinctively tried to use the phone's screen as an input device. The tested users commented that touch screen functionality would make the desired tasks faster and easier. The desire to have touch screen capability most likely arises from two sources. First, there are a large number of touch screen phones on the market today, such as the Treo line and the iPhone. The test users might have expected the new Google mobile phone, whose hardware is not yet on the market, to have capabilities similar to those of current smart phones. Second, the use of mapping in the interface could have led the user to believe that the buttons shown on the screen were themselves actual, functional buttons. It was thought that pictures of the D-Pad with different functionalities would be a logical connection to the D-Pad buttons. However, users tried to use the pictures of the D-Pads themselves. Creating screens that look less like physical buttons could alleviate this problem. In the real world, however, users will be more familiar with their physical phones that a paper prototype. It would be unlikely that a user would try to employ touch screen commands on a phone that he knows does not have such functionality.

All users attempted to use the home key to return to the BorgTV home screen and, instead, reached the Android home screen. This is due to the architecture of the application itself. This was most evident when users were transitioning from one task to another. At the end of a task, a user would generally be at a D-Pad function set. For the next task, the user needed to access a new location, device, or TV guide. Users generally did not want to trace back through the hierarchy to reach the next task's starting point and would press home to start fresh. However, doing so caused the Android home screen to appear. The next iteration of this interface will address this problem in two ways. The next version of the interface will involve a simplified hierarchy, which will be discussed below. Also, easy access to the BorgTV home screen will be available throughout all levels of the interface. It is expected that these two changes will lessen the users need to start again from scratch, and when he does, he can chose the home screen directly from within the application.

It became clear from this experiment that the navigation system and application hierarchy was critically flawed. Users thought that they had to navigate too many screens in order to perform simple television tasks. For example, in the first task,the user had to chose a location, room, device, and then function set. In order to change a channel the user had to navigate between 4 screens. Dirk especially found this frustrating and too time consuming. The default interface had too many levels of organization. This would be especially annoying for a user who only uses BorgTV for one device in one location. Constantly choosing location, room, and device does not simplify remote control use. Bilbo found himself confused as to where he was in the hierarchy and configured the new television twice in the third task. He did not understand the purpose of setting up an entirely new location and room when he just wanted to change the channel on a new television. The next version of this software will utilize a simplified hierarchy system, which by default will only have 1 location and 1 room. However, the option for users to add new locations and rooms will be provided. For instance, the user would only be presented with the option of creating a new room if configured rooms already exist at the present location. Also, Android phones are equipped with GPS capability. This feature could be integrated into BorgTV, to allow the user to choose pertinent remote locations dependent on their physical location.

The procedure for finding and setting new IR codes needs to be more clearly explained in future iterations of this application. This procedure is somewhat tricky and requires user input. The Android mobile phone can send IR signals to a media device, however the devices send no feedback back to the mobile phone. The user must tell the phone when the tested code is in fact correct. The current explanation of this procedure was found to be confusing. Winifred became clearly frustrated when three consecutive codes were incorrect. She did not know that the prototype had been designed to find the IR code on the fourth try. After three tries, she thought that she had failed and consulted the help screen, which simply told her to keep doing what she was doing. At no time was she told that she was, in fact, doing the procedure correctly but simply not found the correct code. Better feedback and instructions for the user while finding new IR codes will be incorporated into future versions of BorgTV.

During the debriefing, users suggested that hot keys for the interface would have been more useful than the D-pad to select desired buttons. One user mentioned using the number pad as hot keys for the 9-button-screen. Continuing that idea, device functions could be mapped to the QWERTY keyboard. In this type of interface, each device function could correspond to a letter. For example, device power could be map to the letter p on the QWERTY keyboard. A clear key to this type of mapping is vital to the usability of this type of interface. For this type of radical change in the interface, another shorter round of low fidelity prototyping should be employed to get real user feedback on this functionality.

Due to time constraints, we were unable to ask users to test all aspects of BorgTV functionality. For example, favoriting shows, dealing with presented reminders, and recommending shows to friends via text message were not tested by actual users. Our team would also have liked to test more functionality within the BorgTV guide. We wanted to test the searching and sorting functionality within the guide based upon the network. We decided that implementing a sorting functionary within lo-fi prototype to be too time consuming as well.

Users also stated that getting to the device through the hierarchy seemed like a lot of work. However, in the real phone, the location can be automatically chosen for the user through GPS. We were not able to simulate that functionality and the user had to manually choose the location. We would have liked to test how the GPS would have affect the user's experience and only manually select the location if no reception was available.

Appendix

Lo-Fi Prototype Images

Prototype Images

Task Descriptions

Users were handed the following tasks written on half index cards.

Task 0: Demo Task

You're in your bedroom, in your house, you'd like to turn on your television, and set it to mute. Find the appropriate device, then perform the actions listed above.

Task 1: Easy Task

You've moved to the living room and would like to control the cable box there. Change the channel on that cable box to 48.

Task 2: Medium Task

You really want to watch Lost on ABC later tonight, but you don't know what time it is on. Set a reminder to watch the show in BorgTV's guide function.

Task 3: Hard Task

You're now at your friend, Bob's, apartment. You would like to control the TV in his living room. Set up the new location, room, and device, then press channel up. His service provider is Comcast.

Demonstration Script

You will be testing our application called BorgTV, which is a universal remote application for your cell phone using Google's Android mobile OS. We've created a low-fidelity prototype, which is obviously made out of paper and markers rather than an actual program and a phone. We choose to create a prototype this way because after getting your feedback, we can easily make dramatic changes to the interface.

We are going to explain briefly how the Android phone works, so that you can treat it as you have been using it for a long time. Right now, we are displaying the home screen of the phone. This button here (points to the menu button) shows the hidden options for an application if applicable. The back button (points to back button) goes back to the previous screen. The home button goes back to this screen no matter what application you are using. The D-pad is used to move the cursor around and select what the cursor is highlighting at the moment.

Dave is going to be our computer. He is going to change the interface like a computer would on the real application. To make the most out of your input, we would like you to say aloud which button you are pressing and explain your thought process as you select the different options within the UI.

Please note we can answer any questions about the process like, “Do I have to actually type a name out, or can I just say it?” However, we cannot answer questions about the UI interface such as “If I press this button, what will it do?” We want to see if the UI is easy enough to use that you can figure it out. If you feel frustrated at any point, please feel free to say that. Frustration gives a great sign that something needs to be changed in the UI and stating what frustrates you is valuable information.

So I am going to start a demo to give you an example of how you should go about using the interface and what we want when you to say while going through the interface:

So you’re in your bedroom, in your house, you’d like to turn on your television and set it to mute. Find the appropriate device, and then perform the actions listed above.

I want to select the application. I want to press up (points to the up button on the D-pad), left (points to the left button on the D-pad), enter (points to the center button on the D-pad).

[Computer changes screen so that the nine buttons are shown, with one of the buttons labeled “My House” and the rest are labeled “New..”]

I see the button “My House” and the tasks states that the TV is in my house. So I am going to select that and see where that goes. Up (points to the up button on the D-pad), right (points to the right button on the D-pad), right (points to the right button on the D-pad), and enter (points to the center button on the D-pad).

[Computer changes the screen so that two options are showed, “Living Room” and “Bedroom”]

Since the TV is in the Bedroom, I want am going to select that. Left (points to the left button on the D-pad), enter (points to the center button on the D-pad).

[Computer changes the screen so that two buttons are labeled “Power Ops” and “CH/VOL”]

Since I want to power on the TV, I am going to see where Power Ops take me. Up (points to the up button on the D-pad), up (points to the up button on the D-pad), right (points to the right button on the D-pad).

[The screen changes so the image looking similar to D-pad is shown with the options Power on the center, Mute on the Left, Menu on the top, Sleep on the right, and Input TV/Video/DVD on the right of the D-pad. ]

I see power there. Since the menu looks like the D-pad, I assume that it is a mapping with the D-pad. So I am going to try pressing the center button. (Points the center button).

[TV turns on.]

Okay, so that worked. Now I am going to press the mute button. Left (Points the left button)

[TV is muted.]

Okay, so this is the process that we would like you to go through. Remember, to say aloud your thought process as you go along through the UI.

Informed Consent Form

You are invited to participate in a study of a new Mobile Device application. In order to learn about the users of our software, we desire on site information and experience with individuals interacting with a recreation of what the software may potentially be. To do this, we would like to use you as a "user" who will provide us with specific information related to your experiences.

Any information that is obtained in connection with this study and that can be identified with you will remain confidential and will be disclosed only with your permission. If you decide to participate, you are free to discontinue participation at any time without prejudice. Your participation is completely anonymous and solely for the purpose of testing the design of the application's user interface. Also we will be video recording your interview for our own, note-taking purposes. These videos will be kept in the utmost confidence and destroyed by Tuesday March 11, 2008.

By signing this form, you agree with the information on this form and have decided to participate.

Signature ____________________________________ Date _____________________

Critical Incidents

Winifred:

• Task 1:

• Use of home button to return to application home and was instead returned to Android home. 4
• Desire for touch screen functionality. 3

• Task 2:

• Initial confusion with TV guide layout. 2

• Task 3:

• Confusion associated with setting up a room in a new location. 5
• Minor confusion associated with necessary steps in new device set up. 3
• Difficulty finding proper IR code for television. 5

Dirk:

• Task 1:

• None

• Task 2:

• Lack of page down functionality. 2
• TV guide not listed in menu list. 3

• Task 3:

• Use of home button to return to application home. 4
• Confusion in beginning of set up of new location. 4
• Too many menus and screens necessary to reach basic TV functions. 4

Bilbo:

• Task 1:

• None

• Task 2:

• Unsure of role of three bells icon. 1
• Dialog box with only one option, 1

• Task 3

• Configured device at wrong location, 4
• Tried to configure television without a brand, 2
• Desire for touch screen functionality. 3
• Confusion in location hierarchy. 4

• Debriefing

• Use of home button to return to application home, 4.

---