PilotStudy-Group:iGroup

From CS 160 User Interfaces Sp10

(Difference between revisions)
Jump to: navigation, search
(Introduction (5 points))
(Other)
Line 35: Line 35:
*: ''Rationale:'' The application name ''iDance'', according to peer feedback from our class presentation, suggested including 'dancing parts.' Others were initially confused as to what our application actually did, and perhaps the name contributed to that ambiguity. Our new application name, ''Choreo'', suggests an application for choreographers and is an unique, catchy name to which we can bind our own definition.
*: ''Rationale:'' The application name ''iDance'', according to peer feedback from our class presentation, suggested including 'dancing parts.' Others were initially confused as to what our application actually did, and perhaps the name contributed to that ambiguity. Our new application name, ''Choreo'', suggests an application for choreographers and is an unique, catchy name to which we can bind our own definition.
-
* '''Implemented persistence: user-created clips and routines are now saved between application runs.'''
+
* '''Implemented persistence: user-created clips, routines, and favorites are now saved between application runs.'''
-
*: ''Rationale:'' Saving data is very important for an application of this type that will require the same clip being used over multiple iterations. It also allows for the user to have a consistent clip that they are using over and over again.
+
*: ''Rationale:'' Saving data is very important for an application of this type that will require the same clip being used over multiple iterations. It also allows for the user to have a consistent clip that they are using over and over again. Saving the favorites data is also critical so that user can quickly and easily access clips and routines that they frequently use.
==Method (10 points)==
==Method (10 points)==

Revision as of 20:01, 21 April 2010

Contents

Introduction (5 points)

The iGroup's application, iDance is targeted towards choreographers and the music managers of music groups. During the course of a rehearsal, the leaders will often spend time seeking towards the correct time in a song. In doing so, they waste time searching and adjusting the track they are on. Time is also wasted waiting for a single song to play in order to reach the appropriate part of the song. Our application will solve this problem by:

  • allowing users to create music segments called 'clips' within a song
  • facilitating the playing/repeating of clips
  • combining clips to form a continuous playlist called a 'routine'
  • allowing easy play-back of all clips and routines

Our pilot study seeked to measure the effectiveness of our current design on real potential users of our application. We set aside 3 tasks for them to accomplish and after a quick introduction, allowed them to explore the interface. The results were analyzed in both a qualitative and quantitative manner and will be used to further the UI of our application. This included: assess the subjective value a user believes our application would have at dance practice, measure the speed with which a user can complete a given task, minimize first-time user errors, and explore the efficiency of our application structure (redundant user exploring).

Implementation and Improvements(15 points)

We have made some design changes with respect to the feedback we received from the demonstration in class. Changes include:

Favorites Tab

Newly added Favorites functionality; clicking the star prompts addition to Favorites. Favorite clips and routines are marked with a colored star ribbon.
  • Clips and routines have a star next to them to allow them to be added to the favorites tab.
    Rationale: Favorites was one of our core functionalities slated from the beginning of the application, but was not yet implemented for the Interactive Prototype assignment.

Routines Tab

  • Playback of routines now available; all clips in a routine play back sequentially.
    Rationale: Routine playback was one of our core functionalities slated from the beginning of the application, but was not yet implemented for the Interactive Prototype assignment.
  • Can now touch-select clip to start playing from within the routine.
    Rationale: From the Lo-Fi and Task Analysis assignments, dance groups highly value being able to manipulate playlists and playback from within playlists. Choosing which clip to play from within a routine is one intuitive way to address this.

Clip Editing Interface

New iteration of the clip editing interface. Scrubbers have been enlarged and given downward hooks to draw the user's attention.
  • Added larger and downwards-pointing sliders to make cutting easier.
    Rationale: 8 peers mentioned in the class feedback that the clip editing scrubbers seemed too small and difficult to manipulate. From our own internal use of the application, we strongly agreed and attempted to create larger sliders. Sliders oriented downwards also draw greater attention to them, instead of allowing the possibility of sliders being lost in the view of the bar.
  • Moved slider up to prevent user from accidentally tapping the tab bar.
    Rationale: By making the sliders larger, the user is presented with a larger area to potentially move the sliders. This area is directly above the navigation bar where the 'done' and 're-name' buttons are. We see a potential problem in which the user is trying to move the sliders, but accidentally hit the navigation bar instead, leading to wasted time.

Other

  • Name of application changed from iDance to Choreo.
    Rationale: The application name iDance, according to peer feedback from our class presentation, suggested including 'dancing parts.' Others were initially confused as to what our application actually did, and perhaps the name contributed to that ambiguity. Our new application name, Choreo, suggests an application for choreographers and is an unique, catchy name to which we can bind our own definition.
  • Implemented persistence: user-created clips, routines, and favorites are now saved between application runs.
    Rationale: Saving data is very important for an application of this type that will require the same clip being used over multiple iterations. It also allows for the user to have a consistent clip that they are using over and over again. Saving the favorites data is also critical so that user can quickly and easily access clips and routines that they frequently use.

Method (10 points)

Participants

Participants were selected based on their affiliations to dance groups; we targeted choreographers and dance group members who led practices and had experience with managing music.

  1. User #1 was the jazz choreographer of a dance team at UC Berkeley. The user did not have experience with an iPhone or iPod, but had experience with basic audio editing using Audacity on a PC.
  2. User #2 was a dancer/hip hop choreographer for a local Berkeley dance group. The user had no experience with an iPhone or iPod, and limited experience with audio editing.
  3. User #3 was a dancer and choreographer for a Santa Clara based cultural dance group. The user possessed both an iPod and an iPhone, but had no experience with audio editing in the past.
  4. User #4 was a coordinator of a dance team at UC Berkeley. The user had both an iPod and a mobile device similar to an iPhone, and had basic audio editing experience with Windows Movie Maker.

Apparatus

User #1 navigating through the song library as part of our playback and navigation task.

The experiment was conducted in informal, public settings: the 6th floor Soda Hall alcove and cafes around Berkeley. Our primary testing equipment was the iPod Touch prototype itself; the user attempted to perform tasks directly on our hardware prototype. We also utilized a digital camera to film the user tests in their entirety, with the camera shooting over the user's shoulder.

Tasks

  1. Your first task is to play back some existing clips and routines we've created.
    • Find one of our Favorite clips and play it.
    • Then, find our routine called "Test Mix" and play it.
    • Then, find our clip "Circle Crescendo" from the song "Circle of Life," and play it.
  2. Your second task is to cut a new clip of your own.
    • Create a clip of the intro to the "Circle of Life," from 0:00 - 0:28, named "Circle Intro," and save it.
  3. Your third task is to create a routine, an assortment of clips.
    • Create a routine named "My Mix", with the clips "Fireflies Asleep" (from the song 'Fireflies'), "Feel The Love Chorus" (from the song 'Can You Feel The Love Tonight'), and "Circle Intro." Save it and play it.
    • On second thought, we didn't mean to add the "Circle Intro" clip. Can you delete it from the routine?

Procedure

User #2 creating a routine as part of our experiment's 3rd task.

Users were first greeted and introduced to the team, then given a quick overview of the goals of the application. We then provided the user a quick demonstration of the application: the facilitator showed the song library, some of the pre-cut clips, the favorites tab, and the routines tab. Though this demonstration touched on each main segment of our application, very little was revealed about how to actually perform the tasks in our user test, preserving the validity of our experiment.

We then revealed each task that our user was to run on our application, one at a time, for our user to try. The user was also given instructions to "think out loud," and to "try and let us know what you think each step you are taking will do." The facilitator of the test was standing by to answer questions when the user got stuck. One group member filmed each user's test in its entirety on a digital camera positioned over the user's shoulder. The other group members took notes of critical incidents and feedback from the user.

After the tasks were finished, all group members asked the user questions about the experience. Questions included specific clarifications of critical incidents (e.g. 'What were your thoughts when you attempted to press this button instead of that one?'), soliciting user opinions of features we were considering implementing in future iterations, and more generic questions, such as:

  • How might you use this application in a typical dance practice?
  • What were some of the features you liked/disliked about the application?
  • Was there any functionality you wished had been present?

Afterwards, the user was thanked for their time and help. The document in the appendix shows the script of how user tests were to be run.

Test Measures (5 points)

For each task we measured:

  • Time spent to finish task - as an overall measure to see how well users can perform the necessary functions
  • # of taps - this gives a more representative account of the amount of actions they performed versus time spent thinking or testing things out. Taps taken while naming a clip was not counted.
  • Wrong/excessive # of taps - we defined this as the number of taps that the user performed minus the minimum number of taps to complete the tasks. It gives us a measure of which areas confused the user the most. We defined the minimum number of taps for each task as:
    • Task 1 = 9
    • Task 2 = 6
    • Task 3 = 12

In Task 2 we also measured:

  • Time spent getting sliders to appropriate position - since cutting is an integral part of the application, it is imperative to make cutting an easy task. We measure the time spent on the sliders to measure the ease of use when cutting a clip.

Results and Discussion (25 points)

User 1 Results

User #1 performing tasks during pilot test on hardware prototype.
Time Taps Excess taps (Taps - min taps) Time spent adjusting sliders (task 2 only)
Task 1 1:40 min 13 4 N/A
Task 2 1:10 min 9 3 0:10 min
Task 3 1:52 min 18 6 N/A

User #1's test went very smoothly. Despite not possessing an iPod or iPhone, the user seemed very comfortable with iPhone navigation bars (top-left meaning "back"), and the user was able to pinpoint key buttons such as "Add Clip To Routine" and "Rename" almost instantly. However, the user did have some slight problems manipulating the clip-edit sliders, despite her small fingers. The keyboard button named "Done" also slightly confused her; she tried to press that "Done" button instead of the alert view's "Save" button while renaming a clip.

The quantitative analysis of User #1 shows that the user had a good grasp on the application as a whole. She did not have many excess actions during the tasks. Although the time taken is a bit long, this is due to the user listening to the music during the tasks. This is shown by the small amount of excess taps during the tasks. Compared to the other users, the user had an easy time with the scrubbers and the application as a whole.

During debriefing, User #1 suggested that moving the right scrubber in the Clip-Edit interface should not restart the clip's play, as it currently does. She mentioned that the "Repeat" cycle icon also looked to her like a "refresh" button. When asked about organizing songs in our Songs Tab, she stated that a hierarchical organization emulating the iPod's might be the best for navigation. She reacted positively to our default clip names during renaming. The user also suggested stop buttons, fade transitions, and clarification on landscape and portrait orientations.

User 2 Results

User #2 performing tasks during pilot test on hardware prototype.
Time Taps Excess taps (Taps - min taps) Time spent adjusting sliders (task 2 only)
Task 1 0:50 min 11 2 N/A
Task 2 1:50 min 45 39 0:42 min
Task 3 1:20 min 20 8 N/A

User #2's test was a little bit rough and hit some bumps. The largest issue was trying to add a clip during Task 2. He spent a lot of time on it and didn't see the 'Add new Clip' button. In the end the facilitator showed him where the button was. Also during task 2, the user hit the 'Done' button when trying to move the right slider and was forced to go back to the play/edit interface. The user spent quite a large amount of time moving scrubbers, which could be the result of his hands being large. The other two tasks went quite smoothly, as the user was able to find the appropriate navigation to finish the tasks.

The quantitative analysis of User #2 shows that the majority of the issues that User 2 had was during task 2. The excess taps were trying to look for the 'Add New Clip' button and considerable time was spent trying to get the sliders to work. By contrast, User 2 had the least excess taps of all the users in Task 1 and was able to complete task 3 in a reasonable amount of time and actions.

During debriefing, User #2 commented that the application was overall easy to use and liked the editing interface. He suggested that instead of having all the songs from the iPod imported into the application, users should be able to import a playlist into the songs category instead.

User 3 Results

User #3 performing tasks during pilot test on hardware prototype.
Time Taps Excess taps (Taps - min taps) Time spent adjusting sliders (task 2 only)
Task 1 0:52 min 12 3 N/A
Task 2 1:27 min 12 6 0:25 min
Task 3 1:52 min 22 10 N/A

User #3's test had a number of very revealing critical incidents for us. During the routine playback section of our 1st task, the user attempted to hit the first table cell to start playback, instead of hitting the play icon. Tapping a table cell to start playback is a feature of the standard iPod interface and something we had not accounted for in our previous prototypes. The user was also sometimes confused by the "Edit" buttons on the top-right, tapping them when she meant to add clips. To address this, we will attempt to highlight our important "Add New Clip" buttons with green color. This user also had a lot of trouble with the touch screen sliders, and on one occasion accidentally hit the "Done" button while manipulating our sliders; this revealed to us that we had a lot more work to do on the clip editing interface layout.

The quantitative analysis of User #3 shows that the user had a few minor errors in each task, as described above. The large number of excess taps in the routine creation task may come from the "Edit" button problem that the user had, and suggests that our interface needs to more strongly highlight how to add a clip to a routine. The relatively long time the user spent on adjusting sliders also reiterates the need for more work on the clip editing interface.

During debriefing, User #3 suggested transitions such as fading and overlap during routine playback instead of direct cuts, which would be too abrupt for a real performance routine. When asked whether the direct cut was enough for simply practice purposes, the user agreed, also agreeing with the assertion that a "prompt-to-play" routine would be useful for practice. The user found our system "pretty self-explanatory" and seemed to approve of many of the planned feature changes we ran by her.

User 4 Results

User #4 performing tasks during pilot test on hardware prototype.
Time Taps Excess taps (Taps - min taps) Time spent adjusting sliders (task 2 only)
Task 1 1:00 min 13 4 N/A
Task 2 1:22 min 12 6 0:23 min
Task 3 2:20 min 22 10 N/A

User #4's test went smoothly overall, but was held up by some errors that the user had a relatively large amount of trouble resolving. Like other testers before us, the "Edit" button in the top-right area of the navigation bar threw the user off when attempting to add a clip. Deleting a clip from a routine was also a difficult task for this user to perform; the user knew to click the "Edit" button, but not to follow through with the iPhone's native red delete button. The user also had a little bit of trouble with the right slider precision (the user hovered around :26 - :30 seconds for some time before finally catching :28 exactly), and like others, attempting to hit the "Done" button on the keyboard instead of the "Save" button on the alert view.

The quantitative analysis of User #4 shows that the user had difficulty during the routine editing, most likely due to the extra navigation accrued by confusion over how to delete a clip. The user was particularly good with the touch screen sliders; the user knew to touch the screen with a small finger area, and she was able to attain control of the sliders easily, but the 23 seconds spent on adjusting sliders means the user had a very difficult time getting to the point in the music she wanted.

During debriefing, User #4 agreed that a manual entry of time fields for clips would have been extremely useful, especially for precision editing. She also suggested that the "Add Routine" button be placed somewhere else in the layout, since the top-left area of the navigation bar is usually reserved for a "back" function. She gave us useful feedback on layout changes we were planning to make (change the color of key buttons, switch playback controls with "Add New Clip" bar, etc.), and also discussed options for managing what music goes into the application's library (choosing Playlists or songs directly from the user's iPod library favorable over loading the entire library).

Overview and Changes for Next Iteration

Supposing that the general dance group population can be representatively sampled by our 4 test users from all sorts of different dance groups, we have come up with some changes we are planning to make in the next iteration of our application:

  • Change clip playback , so that clip continues playing at the current time when dragging the right slider (instead of restarting playback)
    Rationale: Since the right slider controls the end time, it does not affect the status of the music that is being played. As such, the behavior of the player should be consistent with the expectations of the user.
    Severity: 3 - Flexibility and efficiency of use
  • Make the "Add New Clip", "Add New Routine", and "Add Clip to Routine" buttons green
    Rationale: The current black color of the buttons does not stand out and users often did not see it, leading to wasted time and confusion. The green button will have a contrast to the rest of the application to make it more visible to the user. At the same time, green is often linked to addition, which makes more sense since the buttons all create something new. This was suggested by one of the users and agreed upon by multiple. This also helps alleviate the problem of the "Edit" button on the navigation bar, which many users tapped mistakenly when attempting to "Add" new clips or routines.
    Severity:
  • In the play/edit clip interface, move clip start/end times and slider up.
    Rationale: The current position of the song timeline and sliders can be a source of user error as they may accidentally tap the play bar when attempting to move the sliders. One of our users repeatedly pressed the repeat button in her attempts to get the slider to move. By moving it up, we can also expand the target area for the sliders and thus address some of the issues users have been having with moving the sliders.
    Severity:
  • Move "Add (something)" buttons to right underneath the navigation bar
    Rationale: The current position of the 'Add' buttons are on the bottom, which is a spot which the test users did not see immediately. Usually their line of sight was near the top or middle of the screen. Moving it to the top near the navigation bar puts it in a place where users can see it immediately.
    Severity:
  • Move routine play button to the bottom of the screen
    Rationale: Since this is a music application we wanted it to be as consistent with the current iPod's interface, so that users will have to spend less time getting adjusted. Since the regular iPod's play button is on the bottom, our application will follow this choice.
    Severity:
  • Move "Add Routine" button off of the navigation bar and make it consistent with all the other "Add (something)" buttons
    Rationale: One of the users noticed an inconsistency between the interfaces in the 'Add' button positions, which confused them. To resolve this, we shall make all the buttons in the same place, so users do not need to try and memorize different layouts.
    Severity:
  • During routine playback, allow tapping of table cells to immediately start playing the routine from the tapped clip
    Rationale: Similar to the original iPod interface, we want our application to have playback when users tap on a cell. User #3 pointed this our during the test.
    Severity:
  • During routine playback, provide constant feedback on which clip is playing (highlighted table cell) and where in the clip we are (some sort of slider feedback)
    Rationale:Feedback for the user is important, especially within the context of music being played on a playlist. Without feedback users can potentially become confused to which section of the playlist they are in if there are multiple clips of the same song within a playlist. At the same time it allows users to see how much time is left before the next clip and makes explicit what the next clip will be.
    Severity:
  • Allow for direct manipulation of sliders by specifying start/end times.
    Rationale: In task #2, the users were able to move the sliders to the general area of 0:28, but had some trouble making a precise cut. Specifying start/end times insures that a more precise cut can be made with relative ease.
    Severity:

Appendices (5 points)

Raw User Test Videos

Demo/User Test Script

File:IGroup demo script.pdf

Feedback from Class Interactive Prototype Presentations

File:IGroup class feedback.pdf

Critical Incident Logs from User Tests

File:IGroup pilot logs.pdf

Personal tools