LoFi-Group:Jigsaw

From CS160 User Interfaces Fa06

Jump to: navigation, search

Introduction and Mission Statement (6 pts)

Introduction and Purpose

Our project, named Jigsaw, is a software application to be used together with the Anoto pen enabling software and/or Web UI designers to integrate their initial paper sketches of a design onto a computer and into a format ideal for presentations. These objectives were drawn from the concerns, ideas, and viewpoints expressed by user interface designers during the Contextual Inquiry portion of this project. The common points shared by all UI designers during the Contextual Inquiry was the need for a tool that would allow for initial design sketches to be organized, linked, and ultimately shared to clients and/or colleagues as a presentation.

Lo-Fi testing

The Low-Fidelity user testing aims to determine the design flaws in our application while possibly uncovering new ideas and places for improvements to our design. Our specific goals are to discover missing features and incorrectly implemented features. From the user testing we hope to realize how well Jigsaw performs as an ideal application for UI designers.

Mission Statement

Jigsaw aims to provide a comprehensive design tool for user interfaces designers enabling them to integrate their initial design sketches onto a computer to be organized and linked for final output as a presentation to clients and/or colleagues.

Members role in this assignment

Scott Friedheim:Introduction & Mission Statement, Appendix, acted as Greeter/Observer
Patrick Rodriguez: Prototype, informed consent form, acted as Computer
Yen Pai: Method, Discussion write-up, prototype/artifact images, outline notes, acted as Facilitator
Siu Pang Chu: Results and Discussion, acted as Observer
ALL: Developed prototype, conducted user testing, participated in discussion, edited final version of this paper.

Prototype (12 pts)

We use three pages of heavy duty paper to represent our main panels: Sitemap, Workflows, and Wireframes. Only two of these panels are visible at a time, depending on which mode the user selects. Two small pieces of paper representing buttons are placed on the bottom of the Workflow panel to allow the user to switch modes. As the user switches between modes, we move our detached strip of paper representing the menu bar accordingly. A separate tri-folded piece of paper represents the menu options for each menu: File, Edit, and Help.

We use regular sheets of 8.5x11 paper to represent our digital paper. These sheets have three options that can be checked: Sitemap, Workflow, and Wireframe. Checking one of these options indicates that that piece of paper should be imported to the proper panel in the application. As users draw on these sheets, our “computer” draws “thumbnails” on smaller pieces of paper. These small pieces of paper are then laid out on the corresponding panels. In addition, these pieces have two small buttons taped on: Update and Unlink. The Unlink buttons were taped on as a flap in order to only display them when the item actually had a currently associated link.

After the user imports all of the pages into the application, he is ready to associate wireframes to workflows. We use the drawings that the user did on the full sheets of paper to represents what is shown as full screen. If a user double taps on a workflow then we overlay that panel with that drawing. To emulate the functionality of hotspots and highlighting, we use pennies and highlighted strips of paper. When the user selects an item, we place a highlighted strip next to the item. If the user selects an item with many associated items, then highlighted strips are placed next to all of those items. If the user adds a hotspot, we place a penny on the associated area. In our real application, this hotspot will be a transparent, resizable layer. Though we can't replicate the feeling with everyday items, a penny does a well enough job since it can be moved and placed very quickly.

The basic parts of the application

Bag to hold loose parts

Low-fidelity Anoto paper

Thumbnail view

Highlighted items are represented with a strip of colored paper

Zoomed in view using coins to represent "hotspots" (workflow/wireframe view)

Menu dialogues

Thumbnails use folding tab for buttons that appear/disappear according to application state

Index card for pulldown menus (card is folded to display appropriate menu)

Method (12 pts)

Describe the participants in the experiment and how they were selected. Also describe the testing environment and how the prototype and any other equipment were set up. Describe some details of your testing procedure. This should include the roles of each member of the team. To prepare for the experiment, you should assign team members to the different tasks (i.e., greeter, computer, facilitator, observer, etc.) and practice with someone playing the participant. The test measures detail what you looked for or measured during the experiment. You should concentrate on process data (i.e., what is happening in the big picture) in addition to bottom-line data (i.e., time or number of errors).

Following the results of our contextual inquiry, our experiment participants were selected from our pool of target users, software GUI and Website UI designers.

Two of our participants came from the pool of interviewees from our contextual inquiry, with another coming from a referral.

Test Subjects

Marvin Doe: Marvin Doe is an experienced Web/UI designer with a graphic design background and 7+ years of professional experience, currently working fulltime at a video game company as well as maintaining a freelance project designing an interface for an audio plug-in module. At his fulltime job, Marvin manages a Web site that serves as both a marketing site and an online community hub for gamers. Marvin was a participant in our contextual inquiry.

Jane Doe: Jane Doe is an HCI PhD student here at Berkeley. She has experience developing software interfaces and is currently working on a speech driven interface which controls office lighting. Jane regularly works directly with end users on a design to solve problem. Jane was a participant in our contextual inquiry.

Karl Doe: Karl Doe is a software developer at Berkeley. He maintains and designs specialized software for many departments ranging in many different applications. He has been developing software from design to end product for about 2 years. Because Karl designs software for many different kinds of users he always considers the users and their specific needs. Karl's current project involves creating a new web interface off the Berkely.edu website for administration support staff to collaborate.

Team Member Roles

Scott and Siu Pang played roles of Oserver and Greeter; Patrick played the role of the Computer; and Yen played the role of the Facilitator. We thought briefly about rotating the experiment roles but decided that having each team member stay within a role ensure the most consistent results, especially for the computer and facilitator.

Because of scheduling difficulties, it was not possible to assemble the entire team members for every test. In such cases, we conducted the experiment without a greeter and the had the Facilitator fill in as a Greeter.

Test Environment and Setting

We could not maintain one consistent setting for the experiment because it was a higher priority to find cooperative test subjects. As such, the team made an effort to make testing as convenient as possible for each test subject - traveling to the test subject's home or workplace as required. Our experiment had minimal environmental requirements, but we were fortunate to find suitable settings for every experiment.

Below is a description of a typical experimental set-up: We usually set the experiment up on a large table: a large surface area is required due to the size and large number of prototype elements. In addition, our experiment requires the test subject to write and draw on pieces of simulated "Anoto paper."

The test subject sits on one side of the table, across from the Computer (Patrick). The Facilitator (Yen) sits next to the test subject. The Observer and Greeter (Scott, Siu Pang) sit on the same side as the Computer, but slightly removed from the action.

Experiment Description

The experiment begins with the Facilitator giving some background on the application and project. The Facilitator then goes on to demonstrate basic features of the application that will be relevant to the tasks the test subject will be asked to perform:

Create a sitemap/workflows/wireframes
Importing drawings
Linking/unlinking wireframes to workflow elements (creates a movable hotspot)
Switching views (sitemap/workflow and workflow/wireframe)
Saving the project.

After giving the test subject an opportunity to ask questions, the Facilitator asks the test subject to perform a series of tasks using the Jigsaw application. The Facilitator does not give specific tasks like "draw a wireframe using the Anoto pen and import it," but instead asks the test subject to design the initial concept for a simple online bookstore using the Jigsaw application, drawing a general sitemap but providing detail only for the checkout process (beginning with the assumptions that a user is already authenticated - if applicable - and has clicked "check out" from a shopping cart review page). We felt this created a more realistic scenario. We then observe the test subject, paying specific attention to how he/she performs the tasks of interest in the larger context of doing a Web site design.

During the main task, the Computer would observe the test subject's drawings and create, in real-time, "thumbnail" versions of the drawings to more realistically simulate the pen's data import.

After the main task, we end the experiment with a short debrief, asking the test subject for comments on the application, on the process, and asked for suggestions on how to improve subsequent experiments.

Measures and Indicators of Interest

With the exception of the Anoto pen/paper data method, Jigsaw is a very conventional Windows/MacOS application. The interface is not particularly exotic and piggybacked on our test subjects' expert knowledge of existing UI conventions, so we did not expect or observe much user confusion about the interface in general. We were more interested in how the application meshed with our test subjects' design process and whether any crucial features were incomplete or incorrectly implemented.

Experimental Variations

Jane Doe was our first test subject and during the debrief, she suggested that the experiment could be more effective if the somebody on the test team played the role of a client, augmenting our realistic task with client-designer interaction.

In subsequent experiments, the Facilitator would switch between the roles of experiment Facilitator and of a client who wants to establish an online presence for a rare bookstore with search, browse, shopping cart, checkout functionality. In addition to detailing out the checkout process, the test subject was also asked to design a "new inventory" page that would be organize data by "book region." The test subject would interact with the Facilitator (as client) and ask for clarifications on requirements.

Notes and Comments

Our approach of specifying a more "realistic" task embedded with the actual tasks we wanted to study had a drawback in that sometimes it was not always clear to the test subject what we expected - they seemed to think that we expected a specific solution. This was clearest in the first test and in subsequent tests, we minimized user confusion by slowing down the pace of our test (dealing with busy professionals, we were wary of imposing too much on their time) and providing clearer instructions upfront.

Results (12 pts)

Jane Doe

The user testing with Jane Doe began with a brief introduction on our application by the Facilitator; mainly addressing its uses and its target user base. The Facilitator then introduced the functionality of our application by working through a planned example by doing the following: creating a wireframe for a search page of a bookstore website, creating a workflow diagram for how the search page should operate, importing these documents into the application, linking the thumbnails to the wireframes in the application. This demonstrated the basic functionality of the application without going into details of all functionality to be exercised later by the Jane.

After asking Jane Doe whether she found the demo confusing she stated that she was clear on it's workings. From that point we started the user test by outlining the following tasks she should try to accomplish with the Jigsaw system:

Create a wireframe for a shopping cart of a bookstore website.
Add additional wireframes as she saw necessary to make the cart work properly.
For every wireframe that was created she was then asked to draw workflow diagrams.
She was asked to draw a sitemap for the demo bookstore (now containing a search feature and a shopping cart).
Import all of her sketches into the Jigsaw.
Link the sitemap to the workflows & the workflows to the wireframes.

Jane didn't have any trouble making her sketches nor importing her sketches into Jigsaw. However it did take her a few moments after importing to figure out where she should go next; it almost appeared that she just started clicking things just to see what would happen with no clear direction, but this could be due to the fact that a group of people were watching her, making her nervous). One of the very first things she tried to accomplish was to link the buttons on her wireframes directly to other wireframes; this functionality was not available in Jigsaw and demonstrated to the group that it really should be. In that linking process the use of pennies were used as hotspots she didn't like the fact that the hotspots didn't fully cover her drawings under the hotspot and suggested they be expandable; She never tried to re-size them, which was a function Jigsaw supported. While working on the applicaion Jane tried to update her sketches and add new features in the application. She didn't realize that she could only do that to the paper copies of the sketches and then have to update here wireframes/workflows from within the application. This confusion could stem from the fact that Jane was not using a computer and everything was implemented with paper: it was sometimes confusing to her whether she was working on paper or "on the computer."

In the debrief, Jane mentioned that she was confused as to the role of the sitemap, observing that it was almost like a workflow. She suggested that a more effective test might have a group member play the role of the client, thereby allowing the test user to use jigsaw during the concept and presentation stages of the initial design process.

Marvin Doe

To begin, our Facilitator introduced our application, the test plan and the functions to Marvin Doe. Marvin Doe seemed to understand our functions petty well and gave positive feedback for our functions. Then, our Facilitator did a demo. Marvin Doe got confused during the import process. Marvin Doe was questioning how the application identified the paper’s type. He also stated that we needed to make sure there should be title for each pane and that the title should be consistent with the buttons to switch views. He felt that a title for each pane would help him maintain context as he switched from one view to another.

For part 2, we asked Marvin Doe to use our application to draw his idea into site map, workflow and wireframe forms for designing a shopping website. Our facilitator acted as the client and gave him the some requirement of the website. Marvin Doe started the process by taking notes with the Anoto Pen and paper. He mentioned that it would be good if we could provide “notes” type as the new paper option, saying that if he were to use Jigsaw, he would prefer to only work in that environment and not have to switch between Anoto/regular paper. After he wrote down the requirement, he started to draw: 1. site map 2. workflow 3. wireframe. Later on, he started interacting with our application. He was not sure if he should start with the “new project” or the “import” function. After that, he started from the new project function and then imported some drawing into the application by guessing. For five requested tasks:

Create a sitemap/workflows/wireframes: Marvin Doe draw those pictures with Anoto Pen and Paper and checked the associated type box correctly.
Importing drawings: Marvin Doe successfully imported all the drawings. He did it one by one instead of using the function "Import All" because he wanted to see feedback after each import action.
Linking/unlinking wireframes to workflow elements (creates a movable hotspot): Marvin Doe successfully linked several workflow and wireframes together. The "Link" button seem clear enough to show it's function. And he unlink some of them later on.
Switching views (sitemap/workflow and workflow/wireframe): Marvin Doe switched from workflow/wireframe to sitemap/workflow view and did more linking.
Saving the project: Marvin Doe successfully found file->save to save the file.

Karl Doe

We start the test with giving introduction and explanation of our application. Then, we showed a demo to the user.

For the test, we ask Karl to perform the 5 tasks, again asking him to design a Web site for an online bookstore. He started the design process by drawing the picture down in pen and paper. Then, he clicked the "new project" from file to create a new project and “import all” to import his drawings at once. In the time crunch, he forgot to draw a sitemap. He was confused when the application shifted to the sitemap/workflow view because he thought he can see all 3 panes were on screen at once. After a while a searching he found the button to switch sitemap/workflow view to workflow/wireframe view, so that believe he successfully imported all the drawings. He also mention it will be good to get a feedback that how many total drawing have import from the “import all” function, so that he doesn’t need to count one by one to make sure all his drawings were imported.

When he started to connect the wireframe to workflow, he found out that there was a mistake in one of his wireframe drawing. He tried to edit the sketches while it was "digitized", he didn't realize that they would have to edit the sketches and then update. He thought the update process is an automatic process. He didn't immediately realize how to update a sketch that was already imported into the application. So he just try to make a new drawing an import that and delete the old incorrect one. Because he was unfamiliar with Wireframes/Workflows/Sitemaps, user was confused with what pane they were working on; titles would have helped. Finally, he saved the file.

Discussion (12 pts)

Observations and Discussion

The low-fidelity test revealed some minor UI flaws in the original design and suggested several new features in the Jigsaw application:

Issue: When switching between views, it was important for our users to maintain context. A test subject suggested titles for each of the panes.

Discussion: We thought this was a very good suggestion and easy for us to implement and would help with letting the user know where he/she is when after importing drawings.

Issue: All three test subjects expressed a desire to be able to link wireframes directly to the sitemap, as some wireframes were not part of complex interactions that would require a workflow.

Discussion: Again, a very good suggestion. This would require a change to the way a user switches between views. Previously, we had conceptualized only two views: sitemap/workflow and workflow/wireframe. To implement this feature, we would allow the user to select the view for each of the two panes, either via buttons or a pulldown select menu (dependent on space/aesthetic considerations of the GUI layout).

Issue: A "Notes" option on the Anoto paper and a corresponding pane would be useful so that the designer could use one tool and one set of paper for his/her work.

Discussion: We thought the suggestion to integrate a designer's notes into Jigsaw was a natural fit and inline with our mission of trying to integrate the many tools that a Web UI designer uses in the initial concept and approval stages. This would expand the number of panes in Jigsaw to four, but would not add further complexity to the UI because we are now allowing users to select the view for each pane, instead of fixing the application to two views - a view being defined as a fixed set of panes.

Issue: It would be very useful to link "buttons" on the wireframes to other wireframes, so that a designer could step through a wireframe with a client, instead of having to access every wireframe via a workflow.

Discussion: We felt this suggestion was very much inline with our mission, did not add much more complexity to the application, and would allow Jigsaw to accomodate a more wireframe-centric mode of working/thinking. Letting the user choose what is displayed in each pane allows us to implement this feature in a manner consistent with regular "hotspot" creation: 1) the user would select "wireframe" for each pane; 2) on the left pane, the user zoom in on the wireframe that has buttons he/she wishes to make active; 3) the user drags a wireframe from the right pane over to the zoomed-in wireframe on the left, letting go to create a hotspot over the button he/she wishes to make active; 4) the button on the wireframe in left pane is now linked to the wireframe that was just dragged over from the right pane.

Issue: It would be useful for the user to be able to orient the paper: our sample "Anoto paper" was oriented vertically (portrait style) and one of our test subjects mentioned that it might be useful to be able to draw on a piece of paper oriented vertically (landscape style).

Discussion: We felt this was a good suggestion but relatively low priority. We could accomodate this feature by making Jigsaw capable of generating customized Anoto worksheets, a feature which would also make Jigsaw a more complete solution.

Issue: It was confusing whether selecting "New Project" in the File menu was required before beginning to work or import drawings. One test subject asked whether or not the application opened with a new project or if he had to explicitly create one.

Discussion: This was an eye-opening observation since most of our team members and even test subjects more or less took it for granted that a project already existed before importing their work. We decided to have the application open without a new project instantiated and to have all import-related menu items inactive until a project existed. We felt that making the user explicitly create a new project would be less confusing and at the same time indirectly communicate that import functionality only worked when a project was open, without the need of some pop-up dialogue.

Issue: Some test subjects reported confusion as to the value/purpose of the sitemap pane. This had mainly to do with the flexibility afforded by the Jigsaw application: depending on a particular designer's working process, the sitemap could be higher level (broader categorizations) or lower level (down to page-to-page level detail). One test subject noted the fuzzy boundary between a sitemap and a workflow, noting that the only distinction is really in the level of detail.

Discussion: We observed that this was more of a concept/lingo issue, as our more Web-centric UI designers did not display this confusion. We probably could have made clearer the kind of hierarchy we had in mind during the demonstration but were conflicted about imposing a specific conceptual process. We felt, in a sense, that our test subjects might have felt that there was a "right answer" - when really there was just flexibility. One option that we considered merging sitemap and workflows, and allowing nesting of workflows within workflows. However, we felt this resulted in two conflicting conceptual models: the nesting model worked well in the context of a GUI but did not translate well to paper. Ultimately, we decided to retain the sitemap as is, since it best fit the Anoto pen/paper interface.

Issue: Editing of diagrams once imported into Jigsaw would be very useful.

Discussion: It was apparent from the tests that Jigsaw had great potential to become an excellent sketch and concept presentation tool. However, it was also becoming apparent that Jigsaw would become a favored solution only if it was possible for designers to edit the imported diagrams digitally. We did not test this functionality because we tried to focus the experiment on what we felt could be implemented in the given timeframe, but we hold open this possibility and would tag editing capabilities as a high priority on any given implementation schedule.

Concluding Comments and Caveats

The design of Jigsaw represents a constant tension between simplicity and flexibility. We are concerned about overloading the user with too many options and would like to work hard to avoid modes. We feel that we can correct the UI flaws that were discovered and accomodate many, if not all, of the feature requests without adding too much complexity.

The number of features and flexibility of Jigsaw also proved to be slightly problematic in designing a test of reasonable scope. Though the low-fidelity testing proved very useful in flushing out the shortcomings of our initial concept, it could not test how Jigsaw might work in a more dynamic, collaborative environment, where there may be many edits and/or several designers. Given the time constraints for this exercise and of our test participants, testing for these features/attributes was out of scope for the experiment.

Appendix (6 pts)

User Testing

Test subject #1 artifacts

Test subject #1 Observations & Comments
Ratings are at the end of users observations as numbers 1-5 as outlined in the Lo-Fi assignment

After importing their work, the user appeared lost and didn't really know where to go from that point so user just started clicking buttons just to continue doing something (She might have felt rushed to hurry up and click something because of the fact that the group was observing her). (3)
User thought that the workflows were not very useful and that it would be confusing for clients to follow when presented to. (2)
User mentioned the idea of a default hotlink size upon creation of a hotlink. (1)
User didn't really know what a sitemap was nor the use of it. (3)

New Functionality

User suggested linking buttons on the wireframes to branches on workflow directly.
Suggested ability to update workflow on computer (ie. modify/add branches).
Suggested the direct linking of wireframes off the sitemap branches.

Team observations (notes for us):

Perhaps during lo-fi testing, one group member play the role of the client.
Need to slow down intro and let user be clear about what is going on.
Never display all 3 panes at once.
Other than the 2 panes, representing the screen, keep all other stuff away to avoid confusion.
Pay a bit more attention to what should/shouldn't be on the screen during test (ie. there were the 2 buttons on the screen the whole time, wireframe/workflow & workflow/sitemap buttons). This could probably be fixed by just slowing down our computer!
Instead of explaining the process of how a feature would work, let the user do it themselves just as if they were using it to answer their question.

Test subject #2 artifacts

Test subject #2 Observations & Comments
Ratings are at the end of users observations as numbers 1-5 as outlined in the Lo-Fi assignment

User thought the import dialogue was a little confusing (how to choose sitemap for import rather than a wirefram. (3)
User questioned how the application was able to identify which paper type when sketching (ie. sitemap, wireframe, workflow); User was unsure whether the checkmarks at top of sketch papers were for their own use or if the application detects it. (3)
The titles on the workflow, wireframe, and sitemap panes need to have consistent titles. (4)
User was concerned about the inability to edit pages once written on. (1-limitation of Anoto system)
User was confused about how to create a new project; user tried to find it in the import dialogue box; didn't know if there was already a new project already created. (4)
User didn't know why there was a single import function; user thought "Import All" was sufficient. (3)

New Functionality

User showed interest in the ability to take notes with the Anoto pen and have those pages be digitized and added to the application (requires updating checkmark boxes on Anoto paper and creating another pane in the application).
Need to work around the fact that workflow will not always be necessary and therefore should not be present as a pane in the application.
User would like to have feedback after every import
Unlink should be un-highlighted both on workflow and wireframe.
Hotspots need to show that they are floating so that they can be moved.

Test subject #3 artifacts

Test subject #3 Observations & Comments
Ratings are at the end of users observations as numbers 1-5 as outlined in the Lo-Fi assignment

User tried to edit the sketches while it was "digitized"; didn't realize that they would have to edit the sketches and then update. (2)
User didn't immediately realize how to update a sketch that was already imported into the application. (3)
Because user was unfamiliar with Wireframes/Workflows/Sitemaps, user was confused with what pane they were working on; titles would have helped. (4)
At the beginning, the user thought that all 3 panes were on screen at once. (2)

New Functionality

User questioned if having a Sitemap or Workflow was necessary; stated they didn't always use them.
Would like to be able to change default size of wireframe/workflow thumbnails.
User expressed a use in being able to modify/edit sketches once digitized.
User wondered why they couldn't link a wireframe directly from a sitemap.

Computer Response

Below is the functionality of the Jigsaw application and how the computer should respond to a users input commands

Actions for Sitemap/Workflow view:

   * Left click on drawing: display help popup
         o “Drag workflows on right side over to corresponding nodes.”
         o Unfilter workflows
   * Left double click on drawing: same as single click
   * Left click on hotspot: highlight associated workflows with colored border
         o Highlight hotspot with a different color
         o Display “Unlink” button next to each of these workflows
         o Display resize box
   * Left double click on hotspot: filter associated workflows
         o Same “Unlink” behavior as single click
         o Display “Show all” button on Workflow view
   * Left click on workflow thumbnail:
         o highlight thumbnail, all associated sitemap hotspots
   * Left double click on workflow thumbnail:
         o Zoom workflow
         o Shift to Workflow/Wireframe view
   * Click and drag workflow thumbnail:
         o Display transparent thumbnail
         o Release over non-hotspot: create new hotspot that’s associated
         o Mouse over hotspot: highlight hotspot
               + Release over hotspot: associate the two

Actions for Workflow/Wireframe view:

   * Left click on pane: unhighlight everything
   * Left double click on pane: unhighlight everything
   * Left click on workflow: highlight associated wireframes
   * Left double click on workflow: expand selected workflow
   * Left click on expanded workflow: unexpand wireframe
   * Hover over hotspot: Highlight and magnify associated wireframe
   * Left click hotspot: Expand associated wireframe
   * Click and drag wireframe thumbnail:
         o Display transparent thumbnail
         o Release over non-hotspot: create new hotspot that’s associated
         o Mouse over hotspot: highlight hotspot
               + Release over hotspot: associate the two
   * Left click wireframe pane: unhighlight everything
   * Left double click on wireframe pane: unhighlight everything
   * Left click on wireframe
         o If in global view: highlight associated workflows
         o If in expanded view: highlight associate hotspots, if they exist

Informed Consent Form

Retrieved from "http://vis.berkeley.edu/courses/cs160-fa06/wiki/index.php/LoFi-Group:Jigsaw"

Views

Personal tools

Search

Toolbox