PilotStudy-Group:Melody
From CS160 User Interfaces Fa06
Due: before class on Nov 13, 2006
70 Points
Contents |
Introduction (5 points)
The Melody system is a music composition system based in two parts. The first part involves the Anoto Pen system. Users are given a piece of paper with several dot pattern regions. On this paper, users can write the title of musical piece, label it with identifying tags, and are also presented with 8 standard 5-line music bars to compose the music sheet. In addition to just writing notes on the sheet music, users can also specify the instrument for certain bars and can also "connect" bars to play together. The composed music is then stored on a computer and our second part comes into play, a digital database that users can access with an easy-to-use interface that sorts songs by title and tags and allows instant playback. Melody hopes to combine the advantages of a natural input system, namely pen and paper, with the ease of storage, archival, search and organizational functions provided by a computer.
The purpose of the pilot study is to test our interactive prototype on our target group of users. However, the term pilot is used here as this is, in a sense, the experiment to make sure our experiment works (i.e. are we testing the right things, the right variables, are there still major misconceptions about the system?) before we release our project to a larger audience for testing. By focusing on a small group of users right now, we can clear up most problems before our larger test but at less cost to the project group. In our pilot study, we found three users to help us test the system and made sure to carefully note their progress and whatever data they provided to us. The users provided valuable insight into the system, showing us if our improvements from earlier versions were actually improvements and suggesting any new improvements our system might need. The viewpoint is important as the project group vision of the system is almost always different from the a user's vision. The pilot study served as an important step forward in our project, it was the first testing done on our interactive prototype by real users, so it serves as the benchmark from where our project will go.
Implementation and Improvements(15 points)
Implementation
Figuring out where notes are Written
The core of our software is to figure out where users have written down notes and by basing on the position of these written notes to the 5 lined music bar, we can translate this information into a digital musical representation that can be previewed and played back. We have started implementation software algorithms to recognize the position of where the user have written the notes on the score so that these information can be read to produce actual digitally stored music.
We have only worked on the very basic system to recognize the placement of the strokes each time the user makes a mark on the anoto-enabled music sheet.
Figuring out the dot vs the line
In order to figure out the pitch of the music note, we must know where the user have written the notes. From our low-fi prototype testing, we have come to realize that music composers have 2 ways of writing their notes on the score. These are summarized by the diagrams below.
Method 1: the “head” of the note followed by the “tail”, This method is preferred for notes that have a tail that extends “upward”. Usually these are notes that are on the lower octaves of the score.
Method 2: the “tail” first then the head. This method is usually preferred by the computer for notes that have a tail trailing downward, which are usually notes in the upper octaves.
The essential algorithm for is to figure out where is the “head” and where is the “tail” of the note and extract the useful information from both. The head of the note determines the pitch the music and the tail determines the pacing so they are both important. Although note yet fully implemented, we have developed an algorithm to solve this important problem.
This general idea behind this algorithm is derived from the way we observed our users compose the notes. Since the “head” of the note consist of squiggles or a “long stroke which is tightly converged in one region”, we propose to use an area test algorithms to see if many dots/strokes are confined in the same area of the paper, if they are then the average location of these strokes will be used as the “head” of the note.
Since the tail of the note tends to be a line that crosses many bars in the music score, any long vertical line detected will be seen as a “tail”. In order to capture a full note, we will relate the “head” and “tail” by their position and overlap. Initially, we thought that we could just detect the tail as same stroke as the “head”, however in our testing, we found out that some composers actually wrote the “tail” and the “head” of the note in 2 separate strokes.
Although these algorithms have not been fully implemented, we’ve thought about the details of implementation and we will modify our approach as our project matures.
MIDI encoding
In order to encode the music into a playable format, we have chosen to use the ubiquitous, cross-platform format of MIDI. We will be using an open-sourced, java based music synthesizer library (http://www.jsynthlib.org/) to encode our sound into midi files. Although we don’t have an working implementation of this part of the software, we have narrowed it down to the specific software technology we will use and we are in the process of learning the API of the jsynthlib libraries.
Desktop Music Software
We have also made improvements to the music management software to support a “streaming mode” besides the “static batch mode” of writing to the anoto sheet. This is detailed in the design part of this section
Improvements
Design Improvements
Based on the feedback given to our interactive prototype, we have made a host of design changes to our user interface.
The first change we made was to increase the size of the music bars. Our old music bars were about half an inch in height, this gave about a space of 1/10th of an inch for the musician to write each note. The biggest complain we got from our testers was that this was far too small.
We initially chose this size based on normal music scores used by musicians who use paper/pen. However, when using the bulky anoto pen, this seem to have an effect on the user experience. One of our testers commented that the pen seemed to “obstruct” his view of the music bars.
By adjusting the size, printing out the resulting interface and getting feedback from our testers, we found that a height of about 1.2 inches works well for most people. This leaves an ample space of about 0.2 inches per line for the composer to write down notes.
Another feedback we got from our interactive prototype ( from one of our elderly testers) was that the lines on the score marking out the different bars were hard to see. This was mainly due to the fact that the Anoto dot pattern printed onto the paper made the paper “greyish” in order and this decreased the contrast between the paper background and the 5 lines marking out the score sheet. This problem was solved by increasing the thickness of the line.
Another design improvement we have made was to increase the size of the connecting bars which allow users to specify that 2 lines of music should play synchronously. Previously, this was a thin dotted line. We chose to use a dotted line because a dotted line carried the affordance that it can be connected and this was the action which we wanted the user to perform.
However, many users found it hard to draw perfectly straight along a dotted line. To react to this feedback, we have decided to change the dotted line into a dotted region ( the dotted-ness is retained to keep the affordance that it can be “connected” somehow).
Workflow/Usability Improvements
The first workflow improvement we introduced into our program was to introduce a while new “interactive streaming mode” feature into our software. With this feature, the user can compose his/her music on the Anoto enabled sheet and see his/her sheeted music appear in real time on his/her computer screen as its been written.
The decision to implement this feature was actually prompted by users who asked “what if my anoto pen runs out of ink?” and “do I have to keep buying anoto paper? Its expensive!”. Our answer is that the user can now compose music on the same sheet of music with a pen that doesn’t have ink, the caveat to that is that he/she has to be in front of the computer to see the real time feedback from the screen.
This feature is also useful to us programmers as a debug mode in order to test out our various algorithms stated in the implementation section. With this real time mode, we will be able to see if our algorithm/software succeeded/failed without have to go through the complicated process of docking/synching.
The second improvement has to do with an printed instruction telling the user what exactly does the connecting bars (mentioned above in the design section too) in front of the music bars do. This is largely due to the fact that we are offering a new functionality that is not offered by the traditional pen and paper based composition interfaces
Method (10 points)
Participants
User A
- Male, 20 years old, 2nd year Business Adminstration Major
- Writes own music, familarity with writing sheet music
- Recently released his own album
- Basic computer skills, but lacks familarity with music editing software
- How selected: Project group requested his assistance for help
User A was selected since he has a high level of experience with writing his own music. Therefore, his insight into how sheet music is actually written would be invaluable for further revisions to our project. He may be able to lend expert insight into the phsyical creation of sheet music which our project group may lack. He also is representative of our most likely type of user, someone who is well versed in making sheet music but may not be so familar with using computers to support that.
User B
- Female, 22 years old, graduated Art Major, Computer Science Minor
- Took piano lessons until she was 17 years old
- Familar with reading sheet music, some practice writing own sheet music
- Somewhat familar with computers, some experience with music software
- How selected: Project group requested her assistance for help
User B was selected as she has experience with sheet music as well as with some computer music software. Therefore, our project group selected her in hopes of getting two perspectives on this project. One perspective obviously coming from her music background but the other perspective coming from her work with computers. Her music perspective is important for obvious reasons but her computer perspective will be invaluable in helping us figure out if our project is natural and easy to use not just in the sheet music aspect, but also in the interface aspect as well.
User C
- Male, 20 years old, Computer Science Major
- No formal musical training, all self-taught
- Little work with actual paper sheet music, but has some experience
- Mainly works with computer music software for original creations and remixing
- How selected: Project group request his assistance for help
User C was selected for their familarity with computer music software, both on the editing and playback sides. While User C does not do too much work with paper sheet music, he does a lot of work with digital versions of sheet music on the computer, so they have a lot of familarity with how music creation is done on the computer. User C would be able to provide some insight into the standards of digital music creation and how we might able to refine our project further more into line with these standards. Also, User C would be able to provide more insight into the computer aspect than User B may be able to, due to differing expertise.
Apparatus
We did all of our tests in an empty classroom in Soda Hall. The room was fairly nondescript with average room lighting and no markings on the walls/whiteboards. For our testing, we used two laptops in addition to the Anoto Pen. One laptop (laptop A) ran the writing/pen part of our test. Attached to that laptop was our bluetooth connector and the Anoto Pen cradle. Laptop A contained the code for the Melody program and is also the only laptop (to date) that successfully has the streaming functionality of Anoto installed on it. Laptop A is fine working condition with no known issues in its overall functionality and no known issues with the Anoto system. We used another laptop (Laptop B) for the interface aspect of our test. There were no external devices attached to Laptop B. Laptop B contained all the work done with the interface as well as a limited working version of our interface that could be tested on. Laptop B is fine working condition with no known issues in its overall functionality.
Tasks
Composing a short but moderately complex tune utilizing multiple instruments. (Hard)
For this task, users are asked to make an original piece of music. Initially, for organization, users are asked to give their piece of music a title and some tags. After this is achieved, they are then asked to proceed to the actual writing of music. The user will write different letters representing musical instruments in the instrument boxes lined up on the left of the music sheet paper. Then in the connecting region just right of the instrument boxes, the user will "connect" bars of music together, symbolizing that those connected bars will play in parallel. The user also will write the actual notes of music on the provided bars of sheet music.
Making heavy edits to the composition they were to have completed previously. (Medium)
For this task, the user's previously completed composition is reloaded and opened up for editing. However, all actual editing will take place on the previous piece of paper that the user just wrote on. The Melody computer system is loaded up again and this time the user clicks on the "erase" region on the paper to begin the editing their piece. The user then makes marks over notes that they want to remove their music piece. The user then will click on the "write" region on the paper to add new notes of music to their piece as well.
Playing back the composition they just finished via our digital organizing system. (Easy)
For this task, the user is presented with the computer interface aspect of our program and asked to browse through the system and find the piece of music they created from the database. The database is preloaded with other pieces of music in order to make the finding of their piece of music a much more realistic task. The user is asked to for their music by looking for its name through all the pieces of music first (the default setting) and then looking for it by using some of the tags they created to help refine their search. After the music is found, they are asked to play it through the interface.
Procedure
We began our testing by meeting the participant at the predetermined classroom in Soda. Similar to our tests of the low-fi prototype we started off with introductions and some cordial greetings. We also started off with friendly chatting on things unrelated to the upcoming testing to put the participant at ease. We did this to decrease their nervouness and give the participant a more natural atmosphere in which test in.
When the test was officially ready to begin, we presented the participant with a brief demo our Melody system. We showed them in general how the system worked, both on the pen and software aspects. We started off by explaining to the participant the basic way to make a piece of music. We avoided showing them how to actually do the task, but rather, we described to them through words (for example, "You would write the title in this box" or "You make lines here to connect bars of music") how to do certain things on the piece of paper sheet music. We explained to them the functions and purposes to all parts of the paper sheet music. We tried to find a balance in our explanations of the functions of the different pieces of the paper, we did not to be too ambigious, but we did not want to give away the answer (in a sense) to our participant. For the most part, our participants seemed to get our idea, but if they asked questions at this point for clarification, we politely declined to answer at this point, asking them to go ahead with their best guess during the actual testing. We also explained the computer interface to our users as well, again highlighting the functionalities/purposes of each part of the interface, but leaving the actual execution to the user, as that was our actual test of the system.
We also provided some additional comments to the participant as well. We told the participant that they would be observed by our project group and notes would be taken, but to not worry/take notice of us during the testing. They were also told to be as vocal during the testing process, so this meant explaining why there were doing the things they were doing and to inform us of any cool/confusing features of the project. The participant is also informed about the "virtual help" function, meaning if they were confused about something they could access the help function (i.e. ask the project group). However, they were not to do this as a direct question, but rather to access help like how they would on a real computer program (i.e. topic by topic, rather than asking direct questions)
The participant would be seated in front of a long table, whereupon the two laptops (Laptop A and B) were placed. The project group would be seated behind the participant so that they can see the laptop screens, but not close enough to make the user feel uncomfortable. A clock was set up behind the user but visible to project group so that critical incidents could be logged with their time as well as the overall time a task ran. The participant began seated in front of laptop A, as that is where the testing would begin.
When everyone was settled into their positions, we presented the user with the paper sheet music and told them to begin the first task. When they successfully finished that task, we began the second task. After successful completion of that task, we then began with the third task. The third task required the user to shift their focus over to laptop B. We then presented them with the computer interface and presented them with the final task.
When the final task was completed, we had an overall feedback session. We asked questions about how the subject felt during each task and asked for ratings about specific features. Then we asked for an overall rating of the prototype, and accepted any other comments that he or she had. The subject was then free to leave and the project group would review and discuss our notes from the session.
Test Measures (5 points)
Composing a short but moderately complex tune utilizing multiple instruments. (Hard)
- Writing Tags/Titles
- Time: Time is of obvious importance, we want to see how long it would take users to accomplish this task on average.
- Number of tags written: We checked the number of tags written to see if users would need more tag boxes or if they were given too many.
- Using Instrument Boxes
- Time: Time is of obvious importance, we want to see how long it would take users to accomplish this task on average.
- Number of instrument boxes used: We checked to see how instrument boxes the users used to see if more boxes (and in effect, more bars) would be needed.
- Number of different instruments used: We checked to see how many different instruments were used to see if we provided users with too many or too few instruments to use.
- Connecting Bars of Music
- Time: Time is of obvious importance, we want to see how long it would take users to accomplish this task on average.
- Number of lines used to connect: We checked to see how many lines the users used to see if we may need to expand/shrink the size of the connecting region to avoid the region getting too messy/eliminate unnecessary size of region.
- Length of connections: We checked to see the length of the connections, again to see if we may need to expand/shrink the size of the connection region.
- Writing Actual Music Piece
- Time: Time is of obvious importance, we want to see how long it would take users to accomplish this task on average.
- Number of notes used: We checked to see how many notes the users used to see if we may need to add/remove some bars.
Making heavy edits to the composition they were to have completed previously. (Medium)
- Time: Time is of obvious importance, we want to see how long it would take users to accomplish this task on average.
- Number of erased notes: We wanted to see on average how many notes a user would erase to see how useful the erasing functionality is. In addition, we wanted to see how feasible a "messy" sheet could be if users erased too much.
- Number of added notes: We wanted to see on average how many notes a user would add to see how useful the editing functionality is. In addition, we wanted to see how feasible a "messy" sheet could be if users added too much. We may consider adding extra bars of music if users on average add a lot more.
Playing back the composition they just finished via our digital organizing system. (Easy)
- Time: Time is of obvious importance, we want to see how long it would take users to accomplish this task on average. For this task, we found time to be the main primary variable to test on.
Results (10 points)
Composing a short but moderately complex tune utilizing multiple instruments.(Hard)
The users had no problem understanding how the title and tag boxes are for since they are labeled clearly on the sheet. The average time for the user to figure out how the title and tag boxes work is less than 5 seconds. The users all expressed their surprises when they saw their handwritings showed up on the screen in real time since it was their first time using the Anoto Pen. However, they all made the error of turning the pen when writing which caused the error of the system reading the wrong spot on the paper and display the writings a bit off where they were on the paper.
Using the instrument box presented more challenge for the users. They had to access the help function to figure out what are the abbreviations for different instruments. User C took the least amount of time to figure out the functionality of the instrument box with one minute and half. The reason might be because he is a computer science major and has less musical training, making him faster to adapt to our computer based prototype. User A took the longest to grasp the idea of instrument box with almost 3 minutes since he is not familiar with music editing programs. After he got the idea of instrument box, he said the idea is cool to have such a feature implemented. The average time is about 2 minutes for the users to understand the functionality of the instrument box. User A looked uncomfortable with the idea of using the instrument box and raised the question about how to change the instrument selection.
Using the connection feature is the most challenging for all three users since the idea is foreign to traditional paper sheet music composers. User A took about 3 minutes to understand the feature but thought the feature is too confusing to be practical. User B said she has not seen any feature like this with other music editing software. However, unlike User A’s skepticism, User B thought the idea is cool and suggested us to improve it so it is clearer and more user friendly. For User C, he figured out how the function works in relatively short amount of time. User C made 2 errors using this feature by putting same numbers down at the lines he wanted to play together at first. Then he realized the right way of connecting the lines. User A made the same error but with different kind of notation.
The main part of this test, which was writing the actual notes of music proceeded relatively problem free. However, as with writing the tags and title, all users again encountered a problem in regards to the pen position when writing and how it appears on the screen. This is much larger problem when writing notes, as note position is critical on the sheet music. Users slowly adapted by again adjusting their writing style, although all of them noted that the writing style was somewhat unnatural. User A and B actually went ahead and wrote an actual piece of music, but complained that they were constrained by the fact that they could not write any notes above or below the standard 5 lines provided. User C just wrote random notes throughout his piece, focusing more on testing the system than actually making a piece of music.
Results:
- User A: 11:37 to complete entire task, made 5 errors
- User B: 8:41 to complete entire task, made 3 errors
- User C: 5:20 to complete entire task, made 5 errors
Making heavy edits to the composition they were to have completed previously. (Medium)
This task includes asking the users to use the erasing function of the sheet music interface. User A first did not understand how to erase a note he wrote. He asked questions “how do I erase the note?” and “Do I just cross it out?”. After trying the erasing mode for a while, User A expressed his concern with the messiness of the paper interface with all the editing with pen. He said even though the computer interface reflects the user edits, it is hard to read on the paper with all the pen cross marks. User B and C had the same concern regarding to the paper looking too messy. This task takes User A, B, and C around 5, 3, and 3 minutes respectively. User A made 6 errors when attempting this task. Four of the errors were forgetting to switch from erase mode to write mode or vise versa due to unfamiliarity of the system. User B made three errors when erasing the notes. She also found the mode usage to be somewhat difficult and forgot to change the modes when she wanted to switch from writing to erasing and vice versa. User C made the fewest number of errors, two, but also fell victim once to the problem of mode switching.
Results:
- User A: 5:14 to complete entire task, made 6 errors
- User B: 3:05 to complete entire task, made 3 errors
- User C: 2:49 to complete entire task, made 3 errors
Playing back the composition they just finished via our digital organizing system. (Easy)
This task is easy for the users since they all had experience with music playing software like winamp or itunes. All users took around 1 minute to find their music selection, which we had loaded into our database after the created it. User C made a comment after finding the music saying “It’s cool. Easy to use. I like it.” User A was impressed with the outlook of the computer interface. He said, “Nice design” when he first saw the software. User B had a little bit trouble going back to the previous folder when she first clicked on the wrong folder to look for the song we wanted her to find. There were no major incidents with this task.
Results:
- User A: 0:50 to complete entire task, made 0 errors
- User B: 1:23 to complete entire task, made 1 errors
- User C: 0:49 to complete entire task, made 0 errors
Discussion (15 points)
Though the current incarnation of the MELODY system still does not contain all our desired features, performing the pilot test on what we have now nonetheless revealed some lessons about user behavior and UI design that we will be implementing in the near future.
One of the greatest barriers to having a perfect implementation was the inaccuracy of the Anoto locating system. The camera does not capture exactly what spot the user writes in when the pen is slanted, and given the necessity to have a high degree of accuracy when writing notes on bars, we are at a loss for what to do. Other groups in the class may have run into this issue, especially if they need accuracy in their pen strokes. For the experiment, we had to remind users to hold the pen up straight, and streaming visual feedback allowed them to see their mistakes live. There will be no such feedback once a batch feed version is completed, and we are likely relegated to reminding the user with a warning notice on the sheet paper itself.
The instrument box took on average 2 minutes for users to figure out what it is and how to use it. Once they figure this out, future usages will not take very long, except perhaps to consult help to reminded of the code for a particular instrument. We are satisfied that users who may not understand digital editing tools or the MIDI format at all are able to figure it out in a reasonable amount of time, and will probably not be changing the concept much. We are considering placing a small note in the left margin with a listing of commonly used instruments to reduce the amount of time it takes to relearn instrument codes, but we are also concerned with reducing clutter on the page.
Unfortunately, our connection feature that allows multiple bars to be conjoined did not perform very well with users. Given that users thought that it was confusing and unnatural, we are considering omitting it entirely. The nature of this project is indeed very “ambitious,” and it is probably better to have limited functionality that works well than to have more functionality that is confusing and non-intuitive. It is in trying to capture all of the freedom of paper sheet music that we decided on this idea, and perhaps we should simply omit advanced features that are too difficult to implement well and perfect the simple ones.
On the other hand, the only time there were errors or confusion when using the feature was when users wanted to have chords played by one instrument, since composers naturally enjoy the convenience of being able to write chords on a single bar. If we predefine the scope of the feature to say that it is used to harmonize single note melodies only, there might be less confusion, given that there isn’t anything non-intuitive about simply connecting multiple voices, something that is done normally on paper anyways. In this case, it is the clever user that can “discover” that one can produce chords by using multiple voices.
There were significant errors made by users when attempting to use the erase modes to delete passages, mainly having to do with forgetting which mode they were in. When considering this system, we imagined that users uncertain about the status of the erase mode would just deactivate it to be safe, but it turns out that this is an unrealistically high expectation, and that in this case it is the user that is working with the interface, rather than the interface working with the user. People simply forget. Additionally, it is possible that this kind of erase mode can lead to catastrophic error. Since anything stroked upon while erase mode is activated is deleted, the results we got from users made us cognizant that it is possible and not wholly unlikely that a user could have erase mode on, and accidentally make a stroke over a larger area on the sheet, unpredictably erasing large parts of their work with no current way to retrieve it. In this light, we are considering returning to the modeless ‘X’-out system of erasing notes.
As expected, the playback software works without issue, and is sufficiently simple and familiar that users can figure it out without incident. Unless we discover a new type of user that has radically different habits than the ones we tested, we have no plans to further enhance this part of the system, and instead focus our programming efforts elsewhere.
For the “real” test, our goal is to have a system that has no chance of catastrophic error, and to have nothing that is confusing or difficult to use. From the beginning, our philosophy was to have all functions emulate what composers do on paper, and at times our desire to have greater functionality overrides our desire to create something that is quite simply, easy to use. This means that we will rethink our connection bracket and erase mode features. Indeed, many of our users have remarked that what we have is really “cool,” but “cool” should only be an auxiliary bonus to easy-to-learn and easy-to-use.
This has been our third user test, and we would very much like any future user test not to repeat the negative results we’ve received before. This way, we would be able to focus even more on subtle details like how long it takes for users to learn or perform tasks, rather than learning that we should re-think the entire concept behind a task.
Interestingly, this pilot test has made us consider going back to previous ideas that we thought inferior. From this we have learned that we have not considered enough the negative sides to an unimplemented idea, and that in the future we should more thoroughly think through a concept before imagining it as categorically better than what we already have. We will likely return to old ideas and focus on how to best reduce their drawbacks, instead of reinventing new ones in our remaining time.
Workload breakdown (5 points)
- Roland Carlos (25%) Helped setup the pen system for the testing phase. Provided technical assistance (when necessary) for the first two tested tasks. Helped create the Method page.
- Julius Cheng (25%) Served as greeter and gave instructions to testers. Overseed the experiment execution and made sure users stays on task. Helped create the Discussion page.
- Nankun Huang (25%) Helped setup the interface system for the testing phase. Provided technical assistance (when necessary) for the third task. Helped create the Implementation and Improvements page.
- Cheng-Lun Yang (25%) Served as main test observer and kept detailed logs of each tester. Compiled all the results into the results section and logs section for this page.
Appendices (5 points)
Consent Form
Consent form
1) You agree to participate in this experiment and spend up to 4 hours with the MELODY group. 2) You agree that anyone in the MELODY group is not liable for any damage incurred to you in he process of conducting the experiment.. 3) You agree that you are participating in this experiment purely on an voluntary basis and do not expect any form of compensation from the MELODY group. 4) You can withdraw from the experiment anytime should you feel uncomfortable.
You agree to the above conditions
________________________________ Signature
Script
- Greet user, introduce ourselves,
- Re-explain our project and system (they were already given an explanation before agreeing to participate) and answer any general questions that don't reveal the underlying workings of the system.
- Seat the user in front of the computer/pen/paper setup, and take seats.
- Task 1a: "We would like for you to compose a piece of music on this piece of paper. Remember. the only areas that recognize your pen strokes are the ones with a grainy dot pattern. Also remember to title and tag the sheet."
- Task 1b: "Please select an instrument that each line will be played with. If you're confused, you can consult help by asking us. But treat us like a computer help program, where you must select a topic to ask about, rather than stating a direct question."
- Task 1c: "Also, connect bars to show that they should be played simulataneously. Remember, you can consult help if you need to."
- Allow user to complete task, track time, log critical incidents.
- Task 2: "Now we would like you to edit the composition that you just made by erasing parts of it. Consult help if you need to."
- Allow user to complete task, track time, log critical incidents.
- Task 3: "Your composition has been uploaded onto the computer. We would now like you to use this browser to find the song you wrote and play it."
- Allow user to complete task, track time, log critical incidents.
- Thank user, ask for overall feedback.
- Aggregrate logs and data, discuss.
- HELP- Title (never used): "Write a name for the composition in the title field."
- HELP- Tags: "Write one to four key words that are relevant to the composition."
- HELP- Instrument Selector: "Write the single-letter code of the instrument that you want the bar to play with in the square box next to it. P or leave blank for piano, G for guitar, T for trumpet, and F for flute."
- HELP- Combining bars: "Draw a bracket spanning the bars that you want to combine in the long rectangular field left of the bars."
- HELP- Erasing: "Mark the erase mode activator field on the bottom left to activate erase mode. Make any kind of marking on an existing music note to erase it. Mark the erase mode deactivator on the bottom right to stop erasing."
- HELP- Browser (never used): "Find your composition either by title or narrow down the search by selecting a tag on the left window."
Raw Data/Critical Incidents Logged
Writing Tags/Titles: (Average time for task completion – 0:43/Standard Dev: 6.16 seconds)
User A: 0:44 to complete task
Wrote 3 tags
Number of errors: 1
- 0:05: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Thought the streaming feature and handwriting showing up on computer was “cool”
User B: 0:50 to complete task
Wrote 3 tags
Number of errors: 1
- 0:04: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Thought the streaming feature and handwriting showing up on computer was “cool”
User C: 0:35 to complete task
Wrote 2 tags
Number of errors: 1
- 0:05: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Thought the streaming feature and handwriting showing up on computer was “cool”
Using Instrument Boxes: (Average time for task completion – 2:27/Standard Dev: 40.0 seconds)
User A: 3:15 to complete task
Used all eight instrument boxes
Used two different instruments
Number of errors: 1
- 0:35: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Accessed help function to ask what instruments existed and what were the abbreviations for them.
- Expressed frustration at the fact that he has hold pen a certain way to get it to work properly.
- Found the idea of an instrument box “cool”
User B: 2:30 to complete task
Used all eight instrument boxes
Used three different instruments
Number of errors: 1
- 0:15: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Accessed help function to ask what instruments existed and what were the abbreviations for them.
User C: 1:37 to complete task
Used all eight instrument boxes
Used four different instruments
Number of errors: 1
- 0:17: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Accessed help function to ask what instruments existed and what were the abbreviations for them.
Connecting Bars of Music: (Average time for task completion – 1:56/Standard Dev: 48.4 seconds)
User A: 3:03 to complete task
Used 4 bars, all of his connections were of two bars length
Number of errors: 2
- 0:15: Wrote letters in the connecting region box, believing that all the “A bars” would play together.
- 0:15: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Continued to express frustration at the fact that he has hold pen a certain way to get it to work properly.
- Found the idea of a connecting region to be very confusing, too confusing to be practical
- Worried that the region could get too messy if the piece is overly complicated
User B: 1:35 to complete task
Used 3 bars, two connections were of two bars length, the other connection was of three bars length
Number of errors: 0
Critical Incidents:
- Found the idea of a connection region to be “cool”
- Believed that the instructions should be more apparent however, hard to assume from first glance how the region actually works
User C: 1:10 to complete task
Used 2 bars, one of his connections was of four bars length, the other was of two bars length Number of errors: 2
- 0:08: Wrote numbers in the connecting region box, believing that all the “1 bars” would play together.
- 0:08: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Believes that his way (writing numbers instead of lines) may be more natural for users, since users can use any symbol they want
- Wonders what would happened if lines cross or user runs out of space
Writing Actual Music Piece: (Average time for task completion – 3:26/Standard Dev: 65.5 seconds)
User A: 4:35 to complete task
Wrote 152 notes (20, 22, 18, 15, 19, 20, 22, 16), average of 19
Number of errors: 1
- 0:09: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Continued to express frustration at the fact that he has hold pen a certain way to get it to work properly.
- Found the idea that notes appear on the computer “cool”
- Found the idea that the handwritten notes will turn into actual music “cool”
- Suggested that the lines regions be expanded to include notes that happen above/below the provided 5 lines (which is common in music)
User B: 3:46 to complete task
Wrote 146 notes (18, 20, 20, 14, 16, 16, 22, 20), average of 18.25
Number of errors: 1
- 0:05: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Again expressed frustration at the fact that he has hold pen a certain way to get it to work properly.
- Found the idea that notes appear on the computer “cool”
- Found the idea that the handwritten notes will turn into actual music “cool”
- Suggested that the lines regions be expanded to include notes that happen above/below the provided 5 lines (which is common in music)
User C: 1:58 to complete task
Wrote 143 notes (20, 20, 16, 18, 20, 16, 17, 16), average of 17.875
Number of errors: 1
- 0:11: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
Critical Incidents:
- Again expressed frustration at the fact that he has hold pen a certain way to get it to work properly.
- Found the idea that the handwritten notes will turn into actual music “cool”
User A:
- Total time for all tasks: 11:37
- Total number of Errors: 5
User B:
- Total time for all tasks: 8:41
- Total number of Errors: 3
User C:
- Total time for all tasks: 5:20
- Total number of Errors: 5
Making heavy edits to the composition they were to have completed previously. (Medium)
This task includes asking the users to use the erasing function of the sheet music interface. User A first did not understand how to erase a note he wrote. He asked questions “how do I erase the note?” and “Do I just cross it out?”. After trying the erasing mode for a while, User A expressed his concern with the messiness of the paper interface with all the editing with pen. He said even though the computer interface reflects the user edits, it is hard to read on the paper with all the pen cross marks. User B and C had the same concern regarding to the paper looking too messy. This task takes User A, B, and C around 5, 3, and 3 minutes respectively. User A made 6 errors when attempting this task. Four of the errors were forgetting to switch from erase mode to write mode or vise versa due to unfamiliarity of the system. User B made three errors when erasing the notes. She also found the mode usage to be somewhat difficult and forgot to change the modes when she wanted to switch from writing to erasing and vice versa. User C made the fewest number of errors, two, but also fell victim once to the problem of mode switching.
User A: 5:14
Number of erased notes: 16 notes
Number of added notes: 14 notes
Number of errors: 6
- 0:16: Held pen incorrectly, which led to pen stroke not showing up correctly on the screen
- 0:20: Did not enable erasing mode correctly (pen stroke did not register)
- 0:55: Left pen in erase mode when wanting to add a note
- 1:35: Left pen in writing mode when wanting to erase a note
- 2:54: Left pen in erase mode when wanting to add a note
- 3:55: Left pen in writing mode when wanting to add a note
Critical Incidents:
- User asked about correct format for erasing notes.
- User expressed concern about messiness of paper when erasing notes. Feel pencil would be easier since you can cleanly erase notes rather than marking them up with more pen.
- Found the mode switching confusing, found it difficult to remember what mode they were in without constantly checking the computer.
- Continued to express displeasure with the way he had to hold the pen to make sure the strokes were in the correct location.
User B: 3:05
Number of erased notes: 14 notes
Number of added notes: 10 notes
Number of errors: 3
- 0:48: Left pen in erase mode when wanting to add a note
- 1:22: Left pen in writing mode when wanting to erase a note
- 2:01: Left pen in erase mode when wanting to add a note
Critical Incidents:
- User expressed concern about messiness of paper when erasing notes. Feel pencil would be easier since you can cleanly erase notes rather than marking them up with more pen.
- Found the mode switching confusing, found it difficult to remember what mode they were in without constantly checking the computer.
- Asked if the erasing marks could be actually erase notes on the computer, rather than having them show up in red.
User C: 2:49
Number of erased notes: 11 notes
Number of added notes: 13 notes
Number of errors: 3
- 0:48: Left pen in erase mode when wanting to add a note
- 1:22: Left pen in writing mode when wanting to erase a note
- 2:01: Left pen in erase mode when wanting to add a note
Critical Incidents:
- User expressed concern about messiness of paper when erasing notes. Feel pencil would be easier since you can cleanly erase notes rather than marking them up with more pen.
- Suggested an actual eraser instead of just marking notes on red on the screen.
Playing back the composition they just finished via our digital organizing system. (Easy)
This task is easy for the users since they all had experience with music playing software like winamp or itunes. All users took around 1 minute to find their music selection, which we had loaded into our database after the created it. User C made a comment after finding the music saying “It’s cool. Easy to use. I like it.” User A was impressed with the outlook of the computer interface. He said, “Nice design” when he first saw the software. User B had a little bit trouble going back to the previous folder when she first clicked on the wrong folder to look for the song we wanted her to find. There were no major incidents with this task.
User A: 0:50
Number of errors: 0
Critical Incidents:
- Complimented the design of the interface and found it relatively simple and easy to use.
User B: 1:23
Number of errors: 1
- 0:45: User picked the wrong folder and the previous folder button was not immediately obvious to them.
Critical Incidents:
- Noted that she had trouble going back one screen when she made a mistake.
- Noted that the design was simple and relatively easy to use, but could use some more visible labeling in the future.
User C: 0:49
Number of errors: 0
Critical Incidents:
- Found the interface “cool” and “easy to use”. Gave their approval for it. Noted that it was simple in design but had all the necessary features.
