LoFi-Group:MELODY
From CS160 User Interfaces Fa06
Writing and Experimentation Guidelines
Introduction and Mission Statement (6 pts)
The system being evaluated is the MELODY music composition system. The system allows the users to use a pen input device to compose music by writing down notes on a standard 5-lined music bars. The composed music are then recognized by a computer system and stored. The system combines the advantages of a natural input system provided by the anoto pen and paper with the ease of storage, archival, search and organization functions provided by a digital computer.
The overall experiment in this project is to see music can adapt to the music sheet interface designed by us. Although this interface is laid out on paper just like the traditional music composition sheet, additional features enabled by this interface such as the check box are no longer learnt affordances of a computer-illiterate music composer (the lowest common denominator which we are targeting). We want to know if these additions will affect the workflow of the music composer in any way. We also want to know if our computer based interface, which allows the user to organize the composed music is user friendly enough.
The experiment we are conducting in the low-fidelity test is to assess the interaction between users and the core input interface as well as the computer based interface. We want to use the low-fid prototype to assess various workflows as well as “design on the fly” and make any changes if we have to. The low-fidelity prototype will allow us to test out different ideas which we have and see how the user responds to it.
The purpose of this project is to combine the ease of use provided by the interface that composers are used to, with the easy previewing and archiving functions of the computer.
Our mission statement is to “Revolutionize the creative process of music composition by giving composers a set of tools which preserve existing conventions while adding enabling technology that makes storage/archival of music more efficient.”.
Team member and roles in low fid prototyping test
Julius
- Greeter & Facilitator
NK
- Computer
Roland & Allen
- Observer
Prototype (12 pts)
The prototype was constructed by drawing out the main elements of the interface with Microsoft Visio and printing them out. The decision to use Visio over the traditional pen and paper approach are listed below
- We are terrible artists and we felt that inaccuracies in drawings would affect the user’s experience in the low-fid prototype test. In contrast, the Visio printouts are almost exactly what the user will see in the final product, this producing a closer user experience.
- The Visio drawings give a more polished look and feel to the user.
- Visio printouts would provide the same affordances as a pen and paper drawing
- Its easier to revise the Visio diagram since we don’t have to “start over” each time we want to make a new iteration
- Visio provides automatic code generation for C# applications based on a interface diagram, this can speed up workflow better.
Our interface is divided into 2 main parts. There is the pen paper composition sheet called the “MELODY Paper System” as well as the computer part called the “MELODY Computer system”. The role of the 2 parts is clear The paper based part’s main role is to capture the user’s input in a natural way. The computer part is responsible for recognition, archival, preview and organization of the compositions.
All the components of our prototype and their functions are outlined in the sketch below.
(warning: it will take a while to load, and you should maximize the image to read all the text in it )
The paper interface that we cut out and used for the actual experiment:
Method (12 pts)
Participants
We selected the participants based on their levels of experience in working with music composition, both on paper and on the computer. When we were trying to find possible test subjects, our ideal group of users would be one who worked mainly only with paper, one who worked mainly on the computer (digital music composition) and one who acted as a "middle ground" between the two extremes (someone who would describe themselves as being comfortable/working with both paper and the computer in regards to music composition). While it took time to gather the appropriate people, we were able to find a group of users that we felt fit these three types.
We refer to the traditional paper user (lack of computer experience) as User A, the user with a lot of computer experience as User B, and the "middle ground" user as User C.
Environment
We conducted our tests on different days for each participant. Two tests were done in one of the Soda Hall computer labs, and one test was done in an apartment. In each case, the test was conducted in an isolated room with a table large enough to fit our prototype materials.
Since we really have two interfaces, the sheet music paper and the playback system, we tested each separately. We simply set up one of the interfaces in front of the user and with the Nankun, the computer, sitting across from the user, and Julius, the facilitator, sat next to the participant. Allen and Roland, who both observed the testing process, found spots near the testee but not close enough to obstruct testing or make the testee feel surrounded and overwhelmed.
Tasks
We selected three tasks for the participants to complete:
- Composing a short but moderately complex tune utilizing multiple instruments. (Hard)
- Making heavy edits to the composition they were to have completed previously. (Medium)
- Playing back the composition they just finished via our digital organizing system. (Easy)
On a side note, one of the most important features of the MELODY system is that the uploaded music should be editable digitally. We did not test this out because the system merely converts the music into a format editable by other tools; we do not provide music editing software of our own. Thus, we elected to focus mostly on our paper interface, and somewhat on our far simpler playback system.
With the harder tasks we provided very specific goals to the users. Not only did they have to compose a tune, they had to complete several subgoals in order to test optional portions of the interface that might otherwise go unused.
So, first, the users were presented with the paper interface, and were asked to not only compose a tune, but to use more than one instrument. They would also have to title the piece, use the tag fields (but they were not told what they were for), and then upload the completed composition.
The second task that they were asked to perform after finishing the first composition is editing the music on the same paper they worked on before. Specifically, we passage a section of music we wanted them to delete and replace with another passage. In the currently planned system, deleting sections of music is done by 'X'-ing out a note or section. Since the assignment specifically asks us not to tell the user how to do this, we only asked them to do this without telling them the proper procedure. After completing editing, they were to upload the music again.
At this point, we are done testing the sheet music interface and move on to presenting the playback system on the computer. The paper sheet music is taken away, and the subjects are informed that they are done with the paper interface and will now be presented with a low-fidelity version of the playback software. They are to navigate through the browser system and find the most recently edited version of their composition and play it back.
Procedure
Our testing procedure was relatively straightforward and adhered closely to advice from the "Prototyping with Tiny Fingers" reading.
Upon meeting with one the subject in either the Soda labs or one of our places of residence, we would greet each cordially and introduce ourselves and have a short period of time to chat about things unrelated to the task at hand in order to make him or her at ease and not overwhelmed by the number of people involved and the grim atmosphere of Soda labs.
When we were ready to begin, we provided a brief summary of the entire MELODY system to the subject, explaining that there would be an interface that models our paper music sheet, and then a model for our computer software. We reiterated the main details of the experiment that were already explained when arranging the appointment: that we would present paper models of the interfaces, that one of the interfaces was meant to simulate a computer, and that the subject is encouraged to be as blunt and honest as possible when giving feedback about the interfaces. The subject was also encouraged to speak as much as possible while using the interfaces.
We would then seat the subject across from Nankun, playing as the computer, with Julius next to the subject, and Roland and Allen seated anywhere that allowed them to fully observe the interface and the user's interactions with it (note that we did not specifically have a greeter as suggested by the assignment and the lo-fi prototyping reading - we all greeted and instead had two observers in order to maximize the amount of acquired information). Nankun presented the sheet music interface, and Julius explained what it was and the first task that the subject was to perform with it, as well as the subgoals that needed to be fulfilled. Also, the subject was told that they could "go on their computer," and call the help feature in the playback software. They were told to consult this virtual help instead of directly asking us questions.
We anticipated confusion resulting from telling them to fulfill a goal without explaining how to do so, but steadfastly reminded the subject that we would not help out in case they did know what to do.
After the first task was completed and "uploaded," the second task was explained. When that was done and "uploaded," the final task was explained. For the final task, the music sheet interface was taken away and the model playback software was presented. We explained that the two copies of the composition they worked on for the previous tasks were uploaded onto this system, and then we gave him or her the final task.
When the final task was completed, we had an overall feedback session. We asked questions about how the subject felt during each task and asked for ratings about specific features. Then we asked for an overall rating of the concept, and accepted any other comments that he or she had. The subject was then free to leave and we the testers would review and discuss our notes from the session.
Test Measures
Since composing music is a task that can take extremely varying times to complete, we did not measure how much time the tasks took. Instead, we focused on logging errors, the subject's feelings while using the interfaces, and the number of times the subject would call the help function.
How to perform certain tasks with our music sheet design were often not "guessable," we antipicated the help feature be used often, particularly for the first two tasks. We logged how often help was called for and how much detail we had to give up before they were able to successfully complete a task.
Overall, our focus was on ease and enjoyment of use, rather than time efficiency, so our results mainly comprises of user comments and feedback rather than numerical data.
Results (12 pts)
In keeping with our hands-off style, we initially gave them just the interfaces and then a description of the task we wanted them to do (one by one, in the order described in the method section). We did not give them any guidance on how to use the paper or computer interfaces. However, we did inform them at the beginning of the test session that a help function was available if they wanted it. Basically, our help function consisted of asking the "computer" for the correct way to do the task.
However, we tried to make the following points clear to them when they asked for help:
- They would try their best to figure it out on their own before asking for help. This was done to see a user's natural tendancy towards a certain action in a given situation. That way we could see what most users would do in this situation and try and see if we could improve our interface through this data. (If they did something incorrectly, we would mark it as a critical event and inform them of the error).
- The help function would at first, try and nudge the user towards the correct way of doing something. We did not want to give up the complete method/answer from the beginning, in order to try and see if we could still gather information on the user's natural tendancy in the situation. While this obviously wouldn't be the true function of the help section (we would want a clear help function obviously), we felt at this time, it was important to test the interface than make a true help function.
- The help function, if asked again, would give up the method, but leave it up to the user to follow the instructions, to see if there may be a misinterpretation in the directions.
The first task: Composing a short but moderately complex tune utilizing multiple instruments
One common question from all users was how to actually represent certain notes as being to a certain instrument. They were also confused as to which instruments were available from the program. In this situation, they did require some assistance from the help function in order to complete this task successfully. However, once this information was given to users, they had little problem in actually writing the notes, as they were all proficient in writing sheet music, so there was little question about how to write the music on the paper.
All users were unclear on the function of the boxes on the left side of the sheet music (they tell the computer what instrument this part of the music is played in), which again was cleared up after some use of the help function.
User A was confused about the use of tags, but this was due mainly to their lack of experience with music on computers and not with the interface.
There were no major problems after users were informed on how to use instruments.
Time to complete task: User A - Approximately 7 minutes, User B - Approximately 5 minutes, User C - Approximately 5 minutes
The second task: Making heavy edits to the composition they were to have completed previously
All users again had trouble in finding the correct method to edit notes from their composition. It was previously discussed in our design meetings that one could erase notes by X-ing them out, but as we found out, a user's natural tendancy is to just scribble over them to signify this. The help function had to intervene with some "error messages" to signify that the note was not truly erased.
The next natural question to the help function was to find out how to actually edit the music, which most users did successfully once they were told the correct method.
There were no major problems after users were informed on how to correctly erase music.
Time to complete task: User A - Approximately 3 minutes, User B - Approximately 2 minutes, User C - Approximately 2 minutes
The third task: Playing back the composition they just finished via our digital organizing system
All users were able to naturally select and play the song they just made on the paper. They all remembered the tags they made for their song, were able to select from the "Select By Tags" menu on the computer, and then were able to properly select from the "Song List". They then naturally clicked the "play" button to play the music.
Time to complete task: User A, B and C - Approximately 1 minute
Free time
We then asked the users if they had anything else left to say about the interface.
All users seemed to think the idea was interesting, which was a good sign.
User A wondered if there was another playback option from the sheet music to the computer. They also inquired about the large amount of white space on the sheet music.
User B wondered how we could delete music we wrote from the interface and how we could edit tags for music we've already wrote as well. They also noticed that we lacked a cancel button during the transfer.
User C asked how we could edit existing songs and tags. They also noticed the lack of a scroll bar in our song list and asked if the help function would actually be fully developed for the real project.
Discussion (12 pts)
From the feedback and observation of the users, we found out some problems with our interface design. First of all, the boxes at the beginning of every line are there for the users to write in abbreviations of the instruments they want to play the music back with. However, at first glance, none of the users know what it is for. We would need some labels to show the users what the boxes mean. Also, users do not know what the abbreviations are for different instruments and their availabilities. We need to include a list of instruments available and their abbreviations for easier access.
Another big problem we encountered is how to erase a section. Every user has different ways to scratch out a section of notes. Some draw a line across, some scribble over the notes. How is the interface going to recognize the user wants to erase the section rather than trying to decipher what note is being drawn is a problem we need to discuss more.
We also noticed that the reactions from the three users are mostly the same. They all have questions on how to edit sections of the music. Even though this can be learned through a few tries, we should make it clear somehow so users will not be confused. The users all had no problem previewing the music they wrote because it is more straight forward. However, it may not be that obvious for other users with different musical background. That is something we need to consider more.
From the common responses of the three users, we conclude that our design covers many different levels within the target group. The common questions raised revealed our design problems and need to be modified. When all three users said they like one feature, that shows the design is successful and should be kept in the next edition of design. In general, the low-fidelity exercise with real users gives us some ideas about the problems of our design. We get to see the interface work in person and it helps us understanding the needs of our target group.
Appendix (6 pts)
Demo Script
- Introduce Group Member Roles
- Introduce Project
- Introduce Purpose of Prototype Testing
- Present Consent Form
- Time for Q&A Before Testing
- Present Interfaces to User
- First Task
- Second Task
- Third Task
- Free Time
- Wrap-Up/Thanks
Task 1 Description
Composing a short but moderately complex tune utilizing multiple instruments.
"We would like you to use the sheet music provided and write a moderately complex piece of music. How you define moderately complex is up to you. In addition, you must use multiple instruments for your piece of music. Make sure to give your music a title and use the provided tag fields as well. When you are satisifed with your music, upload it to the computer."
Task 2 Description
Making heavy edits to the composition they were to have completed previously.
"We would now like you to edit the music that you just finished writing. Please delete a passage of music and replace it with another passage. After you are finished editing, please upload the music again."
Task 3 Description
Playing back the composition they just finished via our digital organizing system.
"Now turn your focus to the playback system on the computer. You will no longer need the sheet music. Please select the most recently edited version of your composition and play it using the provided interface."
Critical Incidents Logged
User A (Paper Inclined):
- I'm not sure what "tags" mean - Cosmetic problem
- I'm not sure what the boxes on the left do - Cosmetic problem
- How do I represent certain notes as being for a certain instrument? - Cosmetic problem
- What instruments are available? - Cosmetic problem
- I like the idea of being able to have multiple instruments on just one sheet of paper
- How do I send the music? (before noticing the button on the computer interface) - I don't agree this is a usability problem
- I made a mistake, what do I do? (unsure about correction) - Cosmetic problem
- User scribbled out the notes instead of X-ing them to erase them - Minor usability problem
- How do I access help? (before noticing the ? button) - Cosmetic problem
- I think it is cool to turn written notes into digital music
- Oh, I get what tags do now, that's cool if they can understand my handwriting
- Does the music just play from top left to bottom right from the sheet music? Or is there another way to playback music?
- Is there a reason for all the white space on the paper?
User B (Computer Inclined):
- What do the boxes on the left do? - Cosmetic problem
- How do I represent certain notes as being for a certain instrument? - Cosmetic problem
- What instruments are available? - Cosmetic problem
- User scribbled out notes instead of X-ing them to erase them - Minor usability problem
- How do I delete music? - Major usability problem
- How do I edit tags? - Major usability problem
- Is there a way to cancel a transfer? - Minor usability problem
- Is there a way to automatically overwrite old versions of a song instead of seeing all the different versions? (space issues) - Minor usability problem
- I like the overall idea of turning written notes into digital versions instantly
User C (Middle Ground):
- What do the boxes on the left do? - Cosmetic problem
- How do I represent certain notes as being for a certain instrument? - Cosmetic problem
- What instruments are available? - Cosmetic problem
- User scribbled out notes instead of X-ing them to erase them - Minor usability problem
- Is preview on the paper and play on the computer different? - Minor usability problem
- How can I edit music I already made? - Minor usability problem
- Is there a way to scroll up/down on the list of songs in our song list? - Minor usability problem
- Will the help be more useful later? - I don't agree this is a usability problem
- How do I edit tags? - Major usability problem
- Pretty interesting idea, just have to work out some of the kinks I guess
Consent form
Consent form
1) You agree to participate in this experiment and spend up to 4 hours with the MELODY group. 2) You agree that anyone in the MELODY group is not liable for any damage incurred to you in he process of conducting the experiment.. 3) You agree that you are participating in this experiment purely on an voluntary basis and do not expect any form of compensation from the MELODY group. 4) You can withdraw from the experiment anytime should you feel uncomfortable
You agree to the above conditions
________________________________ Signature
