PilotStudy-Group:GroupK
From CS 160 User Interfaces Sp10
Contents |
Introduction (5 points)
Our system consists of a virtual pet that users can interact with and attempt to keep healthy. The tasks that are currently supported are giving your pet exercise, feeding your pet, and playing a racing mini-game. The pet is to serve as a metaphor for the level of health of the user, and any exercise or food given to the pet should have a real life counterpart with the user. In our previous implementation of this application, there was some confusion over the purpose of our app, complaints about text-heavy windows, complaints about excessive alert boxes, and questions about how to find nutritional information. We have updated our app to try to solve these problems and the goal of the following experiments was to test the success or failure of our modifications.
Implementation and Improvements(15 points)
- Home/Help: we changed the presentation of the help page to conform with standard iphone interfaces by using the navbar (see Help Screen figure in Appendices).
- Home/Help: by popular demand, we improved the aesthetic format of the help page. Instead of a solid block of text, we have a topics list and useful icon snapshots of the pages in question (see Help Screen figure in Appendices).
- Smarter status: loaded default parameters (ex: average number of steps = 12,000/day, calories = 12 * weight).
- Finding Calories for New Food Items: searching online occurs in application now, instead of quitting and loading external safari like before. (see New Food Search figure in Appendices)
- High Scores: added high score listings in game tab to motivate users to keep their pet healthier and, therefore, run faster in the minigame.
Method (10 points)
Participants
Our subjects range from being in high school to college, times when it is very easy to get distracted from maintaining healthy habits. Our app is focused toward enticing younger people to be health conscious, and no technical skill should be necessary to use it. Our subjects level of technical skill ranges from experienced to beginner, so the results should reveal any assumptions of knowledge we are putting on the user. Some of our participants engage in regular exercise, while others do not. This is appropriate because it tests to what extent this application will motivate people to practice healthy behavior and the general appeal of taking the time to carry out tasks outlined in this experiment. Although some bias will likely exist due to the small sample size, viewing differences in testing results among these various areas of our target group should lend enlightening information in regard to how much we are currently expecting of the user.
1. Ankush Sharda
- Male
- 20 years old
- Indian
- Business Major (currently in Haas School of Business)
- Brother of Business Fraternity Delta Sigma Pi
- Owns an iPhone
I brought iTouch with the PetTrainer application and met up with him at Haas (I got to interview him because my roommate is a member of Delta Sigma Pi) We went inside the main building and did the interview/test at this area where the Haas students just relax (other students were talking around us during the tes). One of the reason I chose to do it at Haas is so he feels home since he is a Business Student and I did not want the test to be too professional.
2. Tarik
- Male
- 17 years old
- Persian
- High School Student @ Tilden Prep private school
- Owns an iPod Touch and is very technically adept for his age
Tarik had a lot of opinions on how to improve our app regarding each of our tasks. Regarding the overall UI design, there were issues with aesthetics but not much with functionality. Though this subject reported that our UI design was much too simple and in need of more images and icons, he was able to easily follow the steps necessary to complete each of our three main tasks. Despite this ease of use he had many areas that he thought we could improve the app.
3. Homeboy
- Male
- 22 years old
- African-American
- Rhetoric/PoliSci double major at Cal
- Exercise enthusiast
Homeboy is not at all familiar with using the iPhone OS, but is conscious of his diet and exercises regularly. He is highly motivated, and this subject was chosen to gague the level of appeal our app has to those not in need of a motivational boost. He is not familiar with this type of interface, so the usability of our app will really come into play, and basic UI problems should become clear.
Tasks
Easy Task: Race Your Pet!
You have been training hard and eating healthy. You want to see your lean, fit pet in action.
Open the "Game" tab, play a 1-player game, and have fun helping your pet jump hurdles while it runs down a virtual track.
Medium Task: Walk Your Pet!
Your pet's status is "angry" because you haven't taken him on a run lately.
Open the Pet Trainer pedometer and go for a quick jog with the iPhone in your pocket.
Afterwards, check the number of steps you've taken and see if your pet's status has improved (this is not necessary since in order to change the status by walking, you need to walk at least 3000 steps and to be happy 12000 steps).
Hard Task: Feed Your Pet!
You just finished your lunch from TC Garden. Please feed your pet by creating a new food entry (if necessary) and selecting it to reflect your real-life meal. Pet Trainer will log the calorie count for you and your pet. Creating new food and checking the total calories is also a part of the task.
Procedure
We first explained what the main purpose of PetTrainer is (an application to help your diet by motivating you to walk through the pet and keep a healthy diet by letting the pet eat the same food as the user). Then show the home page. Change the pet to the participants' names and informations and favorite animal. Also point out that there is a help page, so if the participants ever get confused, he/she can look at the help page for help. We stated that our goals in these experiments were to observe how well they can perform some simple tasks related to our application. We then showed the user our testing guide, gave the subject time to read, and asked if any clarification is necessary.
Apparatus
For two out of three of our tests we used the iPhone simulator on a Mac computer, once in the Mac lab in Dai Hall and once on a Mac laptop. This introduced some complications when using the pedometer for tracking the user's steps. The third interview (Ansun) was done with an iTouch.
Test Measures (5 points)
• Time taken to complete easy task
• Time taken to complete medium task
• Time taken to complete hard task
• Errors or prolonged episodes of confusion throughout using the app
Results and Discussion (25 points)
1. Ankush Sharda
The test was mostly successful. The easy task was to play the 1 player game and finish the race. He was able to figure out to turn the iPhone sideways and start the game (which some people did not do for Low-Fidelity prototype). He also noticed that he was supposed to tap the screen to jump because when the game starts, it tells the user to tap in order to jump over the obstacles. For the medium task, one of the main point was to make the pet happy from mad but in order to do that, the user needs to walk 12000 steps, so the task was just to walk the pet. He figured out to change to the walk tab page and increased the Steps Counter, so it was successful. The hard task was to feed the pet food. I asked him to feed hamburger and apple. He first went to create food section and was able to figure out how to search for calories online through google. However, he thought when he created the food, it automatically fed the pet, so he did not feed the pet hamburger. He was able to find the total calories and calories of apple but had trouble with the hamburger. Even though this was not part of the task I asked him to delete the burger from the food he ate list and he was able to figure out to swipe his finger on burger because he is an experienced iPhone user but it is very difficult for other people to figure this out for first time iPhone users. He also told me that the Home screen is confusing because the i icon takes to Help menu when most iPhone applications take i icon to an edit menu. Therefore, the i icon should be for editing and “Setting” button should take the user to Help menu (and change the button name the “Help”). However, he thought that the application in whole was really cool and especially liked the mini-game and played few more times even though the interview was over. He said that he would definitely be motivated to walk more and eat healthy just to get higher (faster) score on the mini-game.
Easy Task = Less than 15 seconds (Took him about 9 seconds to finish the race), so no problem. Many people were confused in the Low-Fidelity type but much better now.
Medium Task = Less than 10 Seconds. He knew exactly what to do. Even after the test when I asked him what "Reset Steps" button does, he knew it was going to make the steps to 0. He also knew how to find the total steps by going to history (he was impressed by the application so he played around it for a while after the test).
Hard Task = 1 Minute 30 Seconds. He was confused especially because he did not know that the Created Food was not added to the "Today's Food List." Also, he had trouble finding it since it is not in alphabetical order. There were another error because he did not know how to get the total calories for few seconds. Need to make the Today's Food page clearer.
Changes to make:
- Make it more clear in the Home Screen (the i icon and Setting button), so i icon would be for editing and Setting button goes to Help menu (change to Help button).
- Make Feeding new food simple (maybe feed the food when you create the food?).
- Editing of the food to be easier (especially deleting the food from history since there is a huge chance that a person feeds the wrong food to the pet).
2. Tarik
This test was successful in that our subject was able to complete all of our assigned tasks, but he ran into a lot of things that he did not like about our app when using it to complete these tasks. To start the experiment, I explained the general layout of our app for the subject (a tab-bar application where each tab corresponds to executing a different task). Upon opening the app, this subject was instantly confused because of the height/weight/age information juxtaposed with a picture of a dog. He was unsure what was supposed to represent the user and what was supposed to represent the virtual pet. After this bridge was crossed, the subject was able to discern the pet's mood and determine what he needed to do.
Home Page:
Subject was very confused by the concept of the pet taking on the traits of the user. The homepage has the banner "David's Page" across the top and Tarik was unaware of whether David was a placeholder for the pet's name, or if it was the user's name. He suggested that we have a pet specific page and a user specific page. Or perhaps a pet that is assigned to the user based off of the user's specs. Suggested that a list of recent activities should be presented along with a timestamp indicating when that activity was performed.
Feeding Task (Hard):
Completion Time: 2 minutes
Errors: 2
Our subject did not like our list of presets, stating that their was entirely "too much fruit." Which is true, obviously, we have 10 entries of strictly fruit as our presets. He commented that the feedback animation when your pet was fed become repetitive and a terrible bore. He suggested that we use different animations for different foods, at least for the presets. When adding a new food, our subject judged that finding nutritional information online through google was not worth the effort, and that we should provide a list of estimates for common foods. He even mentioned that it would be nice if he could simply rate the food on a scale to estimate himself how healthy the food is, rather than wholly accurate data.
Walk/Run Task (Medium):
Completion Time: 30s
Errors: 1
Having a button labeled "Reset Steps" as the means to end an exercise session caused a hiccup in usability. Our subject commented that we would not have to change the functionality, just the name of the button to something more appropriate, like "log session." Evinced feelings of there being too many instances where alert boxes appeared when navigating back and forth between the main exercise page.
Mini-Game Task (Easy):
Completion Time: 30s
Errors: 0
Subject became very excited about 2-player mode, which is not yet implemented. He was able to choose a 1-player game and play it to completion, and then find his score in our high scores list. He thought we could improve the game by adding a level of randomization because as it stands it is too predictable.
Changes to Make:
-Make it more clear that the information regarding height/weight is supposed to represent the user. Perhaps have user traits and pet traits.
-Make starting a new exercising session more explicit, rather than simply being prompted to "Start Walking!" whenever you press the exercise tab.
-More variety in our icons, animations, and images
-Make adding health information for certain foods easier; better to estimate than add much complexity.
-Sync up recent activity list with timestamps.
-Rethink name of "Reset Steps" button.
3. Homeboy
The test results of this experiment went the least smoothly out of the three subjects in our study. Basic components such as the iPhone pedometer were not something the subject knew about before the start of the test. There were many more problems with basic navigation through the windows of our application, and this navigation was much slower. This subject was eventually able to complete each task, but it took him some time to get used to our interface.
Home Page:
Subject was able to understand the metaphor of the pet representing the user and identify the status of the pet.
Feeding Task (Hard):
Completion Time: 5 minutes
Errors: 3
Many problems arose because this subject did not know the nutritional information of a "Hamburger" and decided to google the results rather than estimate like the other subjects. Much of the time for completing this task was taken up by the subject learning what buttons he needed to press to navigate to the window he wanted to access.
Walk/Run Task (Medium):
Completion Time: 1 minutes
Errors: 2
Subject did not know how to end his exercise session.
Mini-game Task (Easy):
Completion Time: 30s
Errors: 0
Subject launched app, played a one player game, and checked the high score list with relative ease.
Changes to Make:
- Make button titles for navigating to previous page uniform.
- Give more immediate feedback for when a user ends an exercise session.
- Simplify the nutritional input for creating a new food
Summary and Analysis of Quantitative Data
Completion Time Statistics
| Subject | Walk/Run Task | Feeding Task | Mini-game Task |
|---|---|---|---|
| Ansun | 10 seconds | 1 minute 30 seconds | 15 seconds |
| Tarik | 30 seconds | 2 minutes | 30 seconds |
| Homeboy | 1 minute | 5 minutes | 30 seconds |
Error Statistics
| Subject | Walk/Run Task | Feeding Task | Mini-game Task |
|---|---|---|---|
| Ansun | 0 | 2 | 0 |
| Tarik | 1 | 2 | 0 |
| Homeboy | 2 | 3 | 0 |
There was a lot of variation throughout the data across test subjects, but there was similiar results across tasks for individual subjects. This let's us see that we are depending too much on knowledge of how the iPhone OS works in our users. The time to completion and the error rate for the feeding task was the greatest for each of our subjects. We must make this interface easier to navigate and provide more feedback and guidance for users. The mini-game gave none of our subjects any problems. This perhaps suggests that our mini-game is too simplistic, but it is hard to see how augmenting the features of the game could complicate beginning and executing this task. Some naming issues were a common cause of confusion in the exercise task, as some users took a considerably longer time than others because they did not know how to proceed. This and the unexpected launching of alert boxes caused 2/3 of our subjects to encounter at least one error during execution.
Conclusion
We learned that the task of feeding your virtual pet causes many problems for inexperienced users. We should make navigation through data easier, such as by alphabetizing the list of preset foods and making the names of buttons more uniform for similar actions. The addition of a in-app browser when a user googles a food was helpful because users were not thrown off when they had to restart after using the internet. However, not all users took this route and instead looked for guidance in the app itself. This implies that we should include nutritional information accessible through Pet Trainer. The exercise task caused similar problems between test subjects, which were not detrimental to the task at hand but were a distraction. We should rename buttons with more appropriate names and rethink our placement of alert boxes throughout navigation. Our mini-game caused no problems and was completed very quickly by all participants. This tells us that our interface design for starting and running a game to completion is easy to use, but short completion times throughout urge us to increase the complexity of our game to engage the users more fully.
Appendices (5 points)
File:Groupk-ps-testingScript.pdf
Figures of Implementations and Improvements
 Before: a solid wall of text. |
 After: topics list. |
 After: detailed help. |
 Before: Confusing New Food Creation step. |
 Before: loads external safari app and actually have to quit and reload PetTrainer app. |
 After: Cleaner New Food Creation step. |
 After: in-app browser for quick searches. |
