PilotStudy-Group:iGroup

From CS 160 User Interfaces Sp10

(Difference between revisions)
Jump to: navigation, search
(Overview and Changes for Next Iteration)
(Participants)
Line 46: Line 46:
# User #1 was the jazz choreographer of a dance team at UC Berkeley. The user did not have experience with an iPhone or iPod, but had experience with basic audio editing using Audacity on a PC.
# User #1 was the jazz choreographer of a dance team at UC Berkeley. The user did not have experience with an iPhone or iPod, but had experience with basic audio editing using Audacity on a PC.
# User #2 was a dancer/hip hop choreographer for a local Berkeley dance group. The user had no experience with an iPhone or iPod, and limited experience with audio editing.
# User #2 was a dancer/hip hop choreographer for a local Berkeley dance group. The user had no experience with an iPhone or iPod, and limited experience with audio editing.
-
# User #3 was '''FILL THIS IN'''. The user possessed both an iPod and an iPhone, but had no experience with audio editing in the past.
+
# User #3 was a dancer and choreographer for a Santa Clara based cultural dance group. The user possessed both an iPod and an iPhone, but had no experience with audio editing in the past.
# User #4 was a coordinator of a dance team at UC Berkeley. The user had both an iPod and a mobile device similar to an iPhone, and had basic audio editing experience with Windows Movie Maker.
# User #4 was a coordinator of a dance team at UC Berkeley. The user had both an iPod and a mobile device similar to an iPhone, and had basic audio editing experience with Windows Movie Maker.

Revision as of 17:08, 21 April 2010

Contents

Introduction (5 points)

The iGroup's application, iDance is targeted towards choreographers and the music managers of music groups. During the course of a rehearsal, the leaders will often spend time seeking towards the correct time in a song. In doing so, they waste time searching and adjusting the track they are on. Time is also wasted waiting for a single song to play in order to reach the appropriate part of the song. Our application will solve this problem by:

  • allowing users to create music segments called 'clips' within a song
  • facilitating the playing/repeating of clips
  • combining clips to form a continuous playlist called a 'routine'
  • allowing easy play-back of all clips and routines

Our pilot study seeks to measure the effectiveness of our current design on real potential users of our application. We set aside 3 tasks for them to accomplish and after a quick introduction, allow them to explore the interface. The results will be analyzed in both a qualitative and quantitative manner and will be used to further the UI of our application.

Implementation and Improvements(15 points)

We have made some design changes with respect to the feedback we received from the demonstration in class. Changes include:

Favorites Tab

Newly added Favorites functionality; clicking the star prompts addition to Favorites. Favorite clips and routines are marked with a colored star ribbon.
  • Clips and routines have a star next to them to allow them to be added to the favorites tab.
    Rationale: Favorites was one of our core functionalities slated from the beginning of the application, but was not yet implemented for the Interactive Prototype assignment.

Routines Tab

  • Playback of routines now available; all clips in a routine play back sequentially.
    Rationale: Routine playback was one of our core functionalities slated from the beginning of the application, but was not yet implemented for the Interactive Prototype assignment.
  • Can now touch-select clip to start playing from within the routine.
    Rationale: From the Lo-Fi and Task Analysis assignments, dance groups highly value being able to manipulate playlists and playback from within playlists. Choosing which clip to play from within a routine is one intuitive way to address this.

Clip Editing Interface

New iteration of the clip editing interface. Scrubbers have been enlarged and given downward hooks to draw the user's attention.
  • Added larger and downwards-pointing sliders to make cutting easier.
    Rationale: 8 peers mentioned in the class feedback that the clip editing scrubbers seemed too small and difficult to manipulate. From our own internal use of the application, we strongly agreed and attempted to create larger sliders. Sliders oriented downwards also draw greater attention to them, instead of allowing the possibility of sliders being lost in the view of the bar.
  • Moved slider up to prevent user from accidentally tapping the tab bar.
    Rationale: By making the sliders larger, the user is presented with a larger area to potentially move the sliders. This area is directly above the navigation bar where the 'done' and 're-name' buttons are. We see a potential problem in which the user is trying to move the sliders, but accidentally hit the navigation bar instead, leading to wasted time.

Other

  • Name of application changed from iDance to XXXXX.
    Rationale: The application name iDance, according to peer feedback from our class presentation, suggested including 'dancing parts.' Others were initially confused as to what our application actually did, and perhaps the name contributed to that ambiguity. Our new application name, XXXXX, suggests an enhancement to the iPod music player interface targeted specifically at dancers, which follows the mission of our application.
  • Implemented persistence: user-created clips and routines are now saved between application runs.
    Rationale: Saving data is very important for an application of this type that will require the same clip being used over multiple iterations. It also allows for the user to have a consistent clip that they are using over and over again.

Method (10 points)

Participants

Participants were selected based on their affiliations to dance groups; we targeted choreographers and dance group members who led practices and had experience with managing music.

  1. User #1 was the jazz choreographer of a dance team at UC Berkeley. The user did not have experience with an iPhone or iPod, but had experience with basic audio editing using Audacity on a PC.
  2. User #2 was a dancer/hip hop choreographer for a local Berkeley dance group. The user had no experience with an iPhone or iPod, and limited experience with audio editing.
  3. User #3 was a dancer and choreographer for a Santa Clara based cultural dance group. The user possessed both an iPod and an iPhone, but had no experience with audio editing in the past.
  4. User #4 was a coordinator of a dance team at UC Berkeley. The user had both an iPod and a mobile device similar to an iPhone, and had basic audio editing experience with Windows Movie Maker.

Apparatus

User #1 performing tasks during pilot test on hardware prototype.

The experiment was conducted in informal, public settings: the 6th floor Soda Hall alcove and cafes around Berkeley. Our primary testing equipment was the iPod Touch prototype itself; the user attempted to perform tasks directly on our hardware prototype. We also utilized a digital camera to film the user tests in their entirety, with the camera shooting over the user's shoulder.

Tasks

  1. Your first task is to play back some existing clips and routines we've created.
    • Find one of our Favorite clips and play it.
    • Then, find our routine called "Test Mix" and play it.
    • Then, find our clip "Circle Crescendo" from the song "Circle of Life," and play it.
  2. Your second task is to cut a new clip of your own.
    • Create a clip of the intro to the "Circle of Life," from 0:00 - 0:28, named "Circle Intro," and save it.
  3. Your third task is to create a routine, an assortment of clips.
    • Create a routine named "My Mix", with the clips "Fireflies Asleep" (from the song 'Fireflies'), "Feel The Love Chorus" (from the song 'Can You Feel The Love Tonight'), and "Circle Intro." Save it and play it.
    • On second thought, we didn't mean to add the "Circle Intro" clip. Can you delete it from the routine?

Procedure

User #2 creating a routine as part of our experiment's 3rd task.

Users were first greeted and introduced to the team, then given a quick overview of the goals of the application. We then provided the user a quick demonstration of the application: the facilitator showed the song library, some of the pre-cut clips, the favorites tab, and the routines tab. Though this demonstration touched on each main segment of our application, very little was revealed about how to actually perform the tasks in our user test, preserving the validity of our experiment.

We then revealed each task that our user was to run on our application, one at a time, for our user to try. The user was also given instructions to "think out loud," and to "try and let us know what you think each step you are taking will do." The facilitator of the test was standing by to answer questions when the user got stuck. One group member filmed each user's test in its entirety on a digital camera positioned over the user's shoulder. The other group members took notes of critical incidents and feedback from the user.

After the tasks were finished, all group members asked the user questions about the experience. Questions included specific clarifications of critical incidents (e.g. 'What were your thoughts when you attempted to press this button instead of that one?'), soliciting user opinions of features we were considering implementing in future iterations, and more generic questions, such as:

  • How might you use this application in a typical dance practice?
  • What were some of the features you liked/disliked about the application?
  • Was there any functionality you wished had been present?

Afterwards, the user was thanked for their time and help. The document in the appendix shows the script of how user tests were to be run.

Test Measures (5 points)

For each task we measured:

  • time spent to finish task - as an overall measure to see how well users can perform the necessary functions
  • # of taps - this gives a more representative account of the amount of actions they performed versus time spent thinking or testing things out. Taps taken while naming a clip was not counted.
  • wrong/excessive # of taps - we defined this as the number of taps that the user performed minus the minimum number of taps to complete the tasks. It gives us a measure of which areas confused the user the most. We defined the minimum number of taps for each task as:
    • task 1 = 9
    • task 2 = 6
    • task 3 = 12

In task 2 we also measured:

  • time spent getting sliders to appropriate position - since cutting is an integral part of the application, it is imperative to make cutting an easy task. We measure the time spent on the sliders to measure the ease of use when cutting a clip.

Results and Discussion (25 points)

   * Results of the tests (1/2 page)
   * What you learned from the user study (1/2 page). Document any changes that you plan to make in your prototype as a result of the study.

User 1 Results

Time Taps Excess taps (Taps - min taps) Time spent adjusting sliders (task 2 only)
Task 1 1:40 min 13 4 N/A
Task 2 1:10 min 9 3 0:10 min
Task 3 1:52 min 18 6 N/A

User #1's test went very smoothly. Despite not possessing an iPod or iPhone, the user seemed very comfortable with iPhone navigation bars (top-left meaning "back"), and the user was able to pinpoint key buttons such as "Add Clip To Routine" and "Rename" almost instantly. However, the user did have some slight problems manipulating the clip-edit sliders, despite her small fingers. The keyboard button named "Done" also slightly confused her; she tried to press that "Done" button instead of the alert view's "Save" button while renaming a clip.

The quantitative analysis of User #1 shows that the user had a good grasp on the application as a whole. She did not have many excess actions during the tasks. Although the time taken is a bit long, this is due to the user listening to the music during the tasks. This is shown by the small amount of excess taps during the tasks. Compared to the other users, the user had an easy time with the scrubbers and the application as a whole.

During debriefing, User #1 suggested that moving the right scrubber in the Clip-Edit interface should not restart the clip's play, as it currently does. She mentioned that the "Repeat" cycle icon also looked to her like a "refresh" button. When asked about organizing songs in our Songs Tab, she stated that a hierarchical organization emulating the iPod's might be the best for navigation. She reacted positively to our default clip names during renaming. The user also suggested stop buttons, fade transitions, and clarification on landscape and portrait orientations.

User 2 Results

Time Taps Excess taps (Taps - min taps) Time spent adjusting sliders (task 2 only)
Task 1 0:50 min 11 2 N/A
Task 2 1:50 min 45 39 0:42 min
Task 3 1:20 min 20 8 N/A

User #2's test was a little bit rough and hit some bumps. The largest issue was trying to add a clip during Task 2. He spent a lot of time on it and didn't see the 'Add new Clip' button. In the end the facilitator showed him where the button was. Also during task 2, the user hit the 'Done' button when trying to move the right slider and was forced to go back to the play/edit interface. The user spent quite a large amount of time moving scrubbers, which could be the result of his hands being large. The other two tasks went quite smoothly, as the user was able to find the appropriate navigation to finish the tasks.

The quantitative analysis of User #2 shows that the majority of the issues that User 2 had was during task 2. The excess taps were trying to look for the 'Add New Clip' button and considerable time was spent trying to get the sliders to work. By contrast, User 2 had the least excess taps of all the users in Task 1 and was able to complete task 3 in a reasonable amount of time and actions.

During debriefing, User #2 commented that the application was overall easy to use and liked the editing interface. He suggested that instead of having all the songs from the iPod imported into the application, users should be able to import a playlist into the songs category instead.

User 3 Results

Time Taps Excess taps (Taps - min taps) Time spent adjusting sliders (task 2 only)
Task 1 0:52 min 12 3 N/A
Task 2 1:27 min 12 6 0:25 min
Task 3 1:52 min 22 10 N/A

Summary of User #3's test.

The quantitative analysis of User #3 shows that ...

During debriefing, User #3 suggested, mentioned, etc.

User 4 Results

Time Taps Excess taps (Taps - min taps) Time spent adjusting sliders (task 2 only)
Task 1 1:00 min 13 4 N/A
Task 2 1:22 min 12 6 0:08 min
Task 3 2:20 min 22 10 N/A

Summary of User #4's test.

The quantitative analysis of User #4 shows that ...

During debriefing, User #4 suggested, mentioned, etc.

Overview and Changes for Next Iteration

We found a lot of these general trends in the user tests. Some users thought this, and other users thought that.

Supposing that the general dance group population can be representatively sampled by our 4 test users from all sorts of different dance groups, we have come up with some changes we are planning to make in the next iteration of our application:

  • Change clip playback , so that clip continues playing at the current time when dragging the right slider (instead of restarting playback)
    Rationale: Foo.
  • Make the "Add New Clip", "Add New Routine", and "Add Clip to Routine" buttons green
    Rationale: Foo.
  • In the play/edit clip interface, swap positions of clip start/end times and slider
    Rationale: Foo.
  • Move "Add (something)" buttons to right underneath the navigation bar
    Rationale: Foo.
  • Move routine play button to the bottom of the screen
    Rationale: Foo.
  • Move "Add Routine" button off of the navigation bar and make it consistent with all the other "Add (something)" buttons
    Rationale: Foo.
  • This is something we are going to change.
    Rationale: And here is why; one of the users said this and this and this, as well as some people from the class presentation feedback.

Appendices (5 points)

Raw User Test Videos

Demo/User Test Script

File:IGroup demo script.pdf

Feedback from Class Interactive Prototype Presentations

File:IGroup class feedback.pdf

Critical Incident Logs from User Tests

File:IGroup pilot logs.pdf

Personal tools