PilotStudy-Group:Group 14-MeganMarquardt
From Cs160-sp08
Contents |
Introduction
Our Android cell phone application is called PinMap. It allows users to put pins at certain locations and specify a comment and category associated with each pin, and the user can share each pin with either just themselves, with a list of friends, or with everyone who has the application. The main idea of this application is to create a form of social networking that is location specific and great for travel especially.
The purpose of this experiment is to test the usability of our application and how well it holds up when users not experienced in the computer science programming background get to play around with it. We want to see if it is fast to use, easy to use, and intuitive to a new user. A participant will be selected at random and we will ask them to complete given tasks while we take notes and observe their workflow.
Implementation and Improvements
Since submitting the interactive prototype of PinMap, we have made huge improvements. Our previous prototype used a map as a background image, where each pin is specified by screen coordinates, as opposed to actual latitude and longitude coordinates. We changed this is in this iteration and used the MapView provided by the Android SDK. The pins are now specified by latitude and longitude coordinates.
With the fully functional map, we felt that the “Browse Viewable” main menu option was unnecessary. We decided that when the map was moved, the centermost pin should always be selected, and so the users can now browse through pins in that manner, by just panning the map. There is now also zooming in and out.
Also, we previously just temporarily stored the pin information in a class containing all the information, but this did not maintain pin information when the application was closed. The pins are now implemented using a database and an adapter class was created that can create pins, delete pins, select pins by category, by sharing level, and by both. Each pin is specified by a row id, which gives each pin a unique identification, which is used throughout the rest of the code, and makes managing the pins much easier.
Commentary from classmates during the presentation also brought up the idea that we should have information display on the screen for whichever pin is currently selected. We added that functionality, showing the title and comment, clipping it if the text was too long to fully display.
Also, on a side note, categories were not previously fully implemented, but it is working now and the user can now search according to certain categories, specify a category for a pin, and change the category for a pin. We included this new functionality in our last task, as is described below, so we could test out this new functionality.
Most of the viewable changes are demonstrated with the juxtopositions of the two below images:
--> 
This shows the change in map implementation, the uniformity among selected and other pins, and the display of pin title and comment clip.
See Appendix for additional documentation concerning the changes in the prototype.
Method
A participant was selected at random at the Free Speech Movement café. She is 20 years old and is a third year Economy major at UC Berkeley. She will be referred to as CK for the rest of this experiment report. After asking her for her permission and having her sign the consent form, we moved to the 1st floor of Moffitt Library to conduct the usability study. The experiment was conducted using the Android emulator on my MacBook Pro, while I took handwritten notes.
I first introduced CK to the Android phone emulator in general, telling her the DPad contained the center enter button and arrow keys. I also explained the Menu, Back, and Home keys. Then I went into some of the basic functionality of PinMap, such as the ability to zoom in and out using the ‘I’ and ‘O’ keys on the keypad. I asked CK not to use the touch screen for experimental purposes.
After showing her the basics, I completed a task in order to simply demonstrate how to use the program. I showed CK how to filter the pins so that only the ones I shared with myself were shown. I went to Menu, clicked the “Filter Display” menu option, and selected “My Pins,” and then reverted back to All Pins so that the experiment would be done over default settings.
After the small demonstration, I asked CK to complete three tasks, one at a time, listed below:
- Easy: Look at the map at the current location and browse the viewable pins in order to view the information about the pin at Strada Café.
- Medium: Add a pin at the current location and insert a comment, set the sharing level, and category for that pin.
- Hard: Search for a pin titled “Soda Hall” and change the category of that pin to “Education”.
While she completed each task, I took notes about her reactions to the system, things she said out loud, and mistakes she made. I also roughly timed each task. If CK asked a question, I tried to avoid answering it, such as when she asked, “Can I press this button?” I just responded with “Try it and see if anything happens.” Or if she said “I’m not sure where I would go to do that,” I responded with something like, “Well if you trying out this application on a brand new cell phone, where do you think the correct button might be? Try that out.”
Once she finished the tasks, I asked her what she liked about the system, and what she thought we could maybe improve on, including what things weren’t obvious or what was difficult to figure out. And of course, I thanked her for participating in the usability study.
Test Measures
The only quantitative measure I took was the rough time estimate of completing each task, because our PinMap application should be really quick to access and to get information out of. I also looked at the number of menus were needed in order to complete a task, and how many wrong menus were accessed when completing a task.
As far as qualitative measures, I looked at her reaction, what she got frustrated at and what she thought was fun in the system. I also looked at where she was looking and scanning, by noticing at the movements of the mouse.
Results
In general, she liked it! When the test finished, she said, “Oo, that was fun!” I’ll go through each task and the test measures for each one.
Start Time: 2:28pm
Easy: Browse to find pin at Café Strada
This seemed to be the hardest task since she wasn’t familiar with the system yet. CK didn’t think of just using the arrow keys on the DPad, so she first went to search. She entered in “Caffe Strada” with the misspelling in Café, and when she pressed Search, there were no found search results. However, she got very confused because we have no feedback for the user if there is no results found, just some text at the bottom that says “0/0 Pages.” She stayed on that page for about 20 seconds, not knowing what to do.
She returned to the map screen after a little hint, and then stayed there for another 10 seconds, unaware of the arrow keys, and going to the main menu a few times. I told her that she could just try out buttons and see what happened. She found out that the arrow keys move the map, and she found the pin quickly, but one centered on the pin, didn’t know how to access the information. I gave the same hint that she could just try out buttons and she found out that she could press the center button to access the information. She said, “Oh, I see what you’re doing here now.” A light bulb seemed to go off, and she had little trouble after this browsing the map and accessing pin information.
Task 1 Finished: 2:32pm.
Medium: Place a new pin
She accomplished this task in about a minute, and actually asked with enthusiasm, “Can I put one on my house?” Naturally, since our system is just for sharing fun information, I let her put the pin at her house instead of the current location. She even used the zoom in/out functionality to get a more precise location for her house. She had some trouble with having to delete the text when filling out the information about her new pin. She just started typing and had to go back and delete the text that said “Enter Comment Here”.
Task 2 Finished: 2:33pm.
Hard: Search and edit an already placed pin
This task was completed the quickest, maybe because she had already accessed the Search menu option in the first task. The task was accomplished in about or less than a minute. The only mistake she made was that she pressed enter when she finished typing “Soda Hall” into the text box, which didn’t complete the search but added a return into the text box. She just deleted the text and tried again, and didn’t have trouble using the back button to get to the previous screen.
Task 3 Finished: 2:34pm.
Discussion
The biggest problem that CK seemed to face in our user interface was the functionality of the center and arrow buttons of the DPad in the very beginning of the experiment. We are assuming that this program will be used by users that have experience with the Maps application that comes with the default Google Android Phone, and made sure that our application has the same basic key mapping, except for the Enter button leading to more information about the pin. Having the same key mapping will just allow the user to transition well between applications and will eliminate the need to learn what different buttons do. I don’t think this certain flaw is a big issue, because CK had no problem with the key mapping after she tried it once. This can also maybe be fixed with a better simple demonstration, especially telling the user that when the arrow keys are pressed, the map follows and pans in that direction.
One of the obvious flaws in our interface was that when inserting a new pin, when CK had to enter in the title and comment about the pin, she had to delete text. So we basically need to just change the “Type Pin Title Here” into ghost text, so that it disappears when the text box becomes active.
One suggestion that CK made was to have some auto-correction or spell-check functionality for the search, like when Google asks, “Did you mean Café Strada?” when the user entered “Caffe Strada”. We could also have a view similar to contacts where there is an auto-complete text box and a listing of pins below the text box and the user can just select the pin from there instead of going to a separate screen. This would reduce the number of screens for the tasks, as well as help the user look to see if they spelled a pin incorrectly, or to see pins are slightly similar.
Things to change in the next iteration of design that were not yet fixed from our classmates’ comments and suggestions were generally not found in the single user pilot study, but are listed below, just for reference and completeness:
- Cluster pins together according either to location or pin title, so that the map doesn’t become crowded when this application is scaled to a large amount of users.
- Spacing on the pin’s information screen is spaced out and distracting, so scaling the text to filling the blank space without getting crowded will be a good idea to implement in the next iteration.
- Add categories to the filter menu, so that the user can look at only certain categories, just like the sharing level.
- Add icons to the menu
- Color code pins according to categories (partially implemented, just a few more details to complete)
- Possibly add a list view of the pins
Appendices
The .zip file linked above contains the following documents:
- Script for testing with user (.pdf) - Scanned notes from the experiment (.jpg) - Typed version of the notes from the experiment, also reproduced below (.txt) - Prototype Changes (with additional screenshots) (.pdf) - Screenshot images, also reproduced below (.jpg's)
Below is the typed up version of my raw notes during the usabiliy experiment, in case the handwriting is difficult to read:
Pilot Notes
2:28pm started
(1) went to search, didn't work
no way to return from that
2:30pm: "Um okay I don't get it"
didn't know to use arrows, didn't know to press enter
"Oh I get what you're doing here"
resumed 2:32
(2) understood target
"Oh!" (from making a mistake on entering pin info)
had to delete text
(3) pressing enter didn't work in search bar
really fast!
2:33-2:34 less than a minute
"super fun!"
like:
better than google maps because it has the comment with the pin
zoom in/out is nice
improvement:
keep wanting to use touch screen
when searching, have some default catches (for misspellings, etc)
Screenshots
Main display and new menu with only 4 options:
Showing zooming in/out:
