FinalPrototype-Group:GGiD

From CS160 User Interfaces Fa06

Jump to: navigation, search

Contents

Team Member Roles in Final Prototype

  • Robert Taylor: Writeup and Firefox extension
  • Vijay Rudraraju: Emailer Design, OCR, Debugging GUI
  • Bryce Lee: Emailing articles, OCR, switching from batch to streaming
  • Simon Tan: Emailing articles, OCR, switching from batch to streaming

Problem and solution overview

Employees in publication-driven professions, such as public relations, have embraced online news sources, avoiding paper sources whenever possible. This technological adoption has moved the respective industries of these professions towards the "paperless office". Yet, for all the affordances the "paperless office" promises, we found dissenters. During our contextual inquiry, a number of users mentioned eye strain and discomfort from reading news from a computer screen all day. Other users would print out news to read when out of the office. Obviously, a niche for paper still exists in the "paperless office", and we aim to help users in this niche by creating a seamless experience between the on-screen world and the paper world using Anoto technology. From a Firefox Extension where users can keep a queue of interesting bookmarked articles, online web articles are processed to be both Anoto and human friendly. Through an R3 based application that translates data written on the paper back to the computer, we bridge the gap between computer and paper in a way that is both flexible and easy to use. With this product, we aim to empower mobile PR like never before, allowing them to be as productive with paper versions of news articles as they are in front of a screen.

Target User Group

Our target user group is composed of public relations professionals, highlighted by our case study participants from the PR firm "Text100" [[1]]. These individuals spend a considerable amount of time scouring online news sites for articles relevant to their clients. While the amount of time spent on this activity is dependent on their position in the company, all individuals participate at some level, expanding the relevance of our solution. For example, an employee with the position of Account Manager in our user group's firm tends to spend the most time online, and thus they are our primary users. At the same time, any of the employees of the firm who commute in some form of shared transportation (carpool, bus, BART, etc) may want to read news articles during the commute, and they, too, would be a user group we are targeting. While we performed all of our project's research at Text100, we feel the project would also be applicable to employees at other PR firms whose jobs entail similar responsibilities.

Tasks

Easy Task: Annotating an Article

Task Summary: Using an Anoto pen, users annotate articles printed with our program. Annotating consists of highlighting any portion of the article or writing comments on the side. While freeform writing is also permitted in the article region through the pen button, the comments region designates a specific region for the annotator to convey his/her thoughts to an audience and likewise for the audience to know where to look for relevant information.

Rationale Behind the Task: We chose this task as it is perhaps the most integral part of our project; physical annotation embodies a key benefit that our solution bridges between the paper and screen environments. We rated this task as easy, since the user should be able to intuitively utilze many of the provided features due to previous experience with pens and paper. All other features not inherently obvious through paper affordances should be attainable with minimal steps and a minimal learning curve, in a strive to preserve the "naturalness" of this task.

Medium Task: Queueing an Article in Firefox

Task Summary: The user is browsing the Internet in Firefox and finds an interesting article. They want to add it to the NewsToGo queue because it warrants in depth annotation and review, but is not a high priority article that requires immediate attention. To add it to the queue the user simply right clicks the context menu and chooses "Add to NewsToGo Queue..." or clicks the equivalent button on the toolbar. They are then presented with a confirmation dialog, and the article is added to the queue.

Rationale Behind the Task: This task is also a very important part of the project - without it there would be nothing for the end user to annotate later. This task is rated at medium difficulty for two reasons; first, some users in our interviews were having trouble adding articles. Second, the subsequent task of printing the articles goes hand in hand with this task and requires the user have set up a 1200dpi printer as the default printer. This could possibly present problems for some users.

Hard Task: Emailing the Article

Task Summary: The user wants to categorize, summarize, and send the article they just annotated. The first step requires listing any perceived categories the article might fall under (categories typically are just which company/client the article is about); the second is just writing a summary on the first page. Finally, to send the article, email addresses are written in the appropriate region and the send box must be checked. Articles are typically emailed back to the user for review before sending to coworkers, or can just be sent out to coworkers immediately.

Rationale Behind the Task: We included this task as it demonstrates a more complex, multi-step task the user will have to complete. Unlike the easy task of annotating the pages, the affordances provided by this part, such as physically assigning digital elements to physically produced annotations, is completely alien. In our usability studies users often had the most problems with this task, even when instructions were included in the printout. For this reason it is ranked as a hard task.

Design Evolution

Changes Between Stages

  • Initial Sketches: Our initial sketches differed dramatically from our final project. After the Contextual Inquiry, as detailed below in the section "What was the most helpful prototype and why?", we revised major elements of the project. Our initial project was intended for users to be able to read an article they had printed out, then "digg" it if they thought it was good-that is, they could rate the article well, and if the article got enough high ratings from people in the company, it might show up on the company's internal website. However, our contextual inquiries unveiled that the majority of potential users did not use paper as their dominant medium to read articles. Also, the idea of printing every single article would be a repetitive and often unnecessary task, putting the users through too many extra steps in order to use our system. Essentially, we scrapped the entire original idea, since even in the conceptual stage, it provided little use and too much hassle to the target user group. Below are sketches from before and after the Contextual Inquiry:

Firefox Extension Changes

  • Low fidelity prototype: The low fideility prototype was the inception of our new project idea. Here, we presented many of the key components, such as the firefox plugin. The plugin form was a key development for our project. From a deployment standpoint, it provided an easy, platform independent installation method. Usage wise, the plugin allowed us to tap into existing user applications, which allowed this portion of the interaction to be an extension of a familiar application, reducing the learning curve involved with a new application. Also, we introduced the Outlook interface, which included organized folders in the side panel to hold drafts. This interface was our attempt to leverage the familiarity of existing applications. However, as described in the later stages, our extended functionality broke the existing metaphors of Outlook and thus was later scrapped.
    • LF-1: The general layout - toolbar buttons and a contextual menu. The introduction of these elements into the web browser was to bring natural feel and usability to the computer side of NewsToGo. Much like the naturalness of annotations to paper, we wanted users to utilize current affordances of the web browser to interact with our solution.
    • LF-2: When an article is added, a prompt comes up regarding adding the page to the queue. If the user clicks OK then a confirmation dialog comes up notifying the user that the article has been added to the queue. Note the "don't show this again" checkmark. This dialog provided an interactive confirmation, reassuring users that their action was completed.
    • LF-3: The initial window for articles that have been queued. Initially articles would be checked and there were alternating colors for rows to better differentiate between the two. The title, date, and first few words of the article would appear on each row.
    • LF-4: Users were notified of when they were about to remove an article. To make things clear the buttons were labelled Remove and Don't Remove.
  • Interactive prototype: The major development under the interactive prototype was the computer interface. Here, we fully developed the firefox plugin to allow the selection and retention of queued pages, along with the page generator to create printable pdfs of the article from just a URL. Also, this stage brought the introduction of the page generator, which took a URL and created PDFs in the Anoto-friendly page layout (see paper interface evolution below) of the linked page. This step also unveiled limitations, such as the inability to firefox to easily call external applications from javascript (most likely due to security risks). We tried to resolve this issue throughout the rest of the further stages (and actually got java applications to be invoked); however the dependencies of R3 prevented a full integration between the two components. Image:Interactive-Prototype-GGiD-NewsToGo.zip
    • IP-1: The general layout - the toolbar buttons now have their own toolbar. The icons are meant to help evince the purpose of the buttons: a plus for adding articles, and the NewsToGo logo for going to the NewsToGo queue. The context menu is in place just like in the lowfi prototye, but can't be shown in a screenshot.
    • IP-2: The adding article dialog is about the same as the low fi prototype except that we removed the second window notifying the user that the article was removed from the queue. This removal was based on low-fi prototyping feedback and validated in the Interactive Prototype by no one mentioning anything about its dissappearance.
    • IP-3: Our implementation of the document queue window stressed our limited knowledge of XUL. Alternating colors for rows seemed impossible, as did checkboxes- listbox, the primary object for implementing the list didn't support these features, and its counterpart richlistbox, which would have supported checkboxes had other shortcomings that made it impractical. Finally, icons were added throughout the extension for a little color and to perhaps aid usability- instead of reading a familiar button, users need just look for the color of the icon associated with the button.
    • IP-4: The Remove/Don't Remove buttons have been changed to OK/Cancel.
    • IP-5: A help window was added as a result of us not being able to do all we could with a listbox. Although testing confirmed that most of our users were already familiar with shift or control-clicking for multiple item selection, we felt some kind of easy to access instructions would be good to have.
  • Pilot Usability Study: Most core functionality was in place at this point; the majority of the changes here were cosmetic. However, this stage significantly influenced later development for the paper interface (see below) Image:Newstogo pilot.zip
    • Fastfacts list more readable: each element in the list now displays what kind of FastFact it is before the FastFact itself [removed when FastFacts actually implemented due to problems caused when trying to remove articles]
    • US-1: NewsToGo button name in toolbar: has been changed to NewsToGo Center make its function more understandable.
    • US-2: Message in dialog box for adding articles changed: it now includes the title of article, and the message itself has been changed to make it more understandable.
    • US-3: Larger and less cramped layout for the main window: about twice as wide, with all the buttons being on one row now.
    • US-4: No change
    • US-5: Help Section: has been made more comprehensive.
    • US-6: Made the FastFacts more Wizard of Oz friendly and less hardcoded: the filtering function and remove functions actually work now; an entry was added to the context menu add FastFacts (again as a Wizard of Oz mechanism).
    • Other
      • Consistency in labels: Any lingering reference to SparklerNotes has been changed to FastFacts.
      • Changed most icons to Windows or Outlook icons and have attempted to match icons for functions in the extension with a similar function they would have played in Outlook, etc.
      • Cleaned up code so function names better reflect what they do and gave xul/javascript proper indenting
      • Continued work on calling a local Java method from the extension: the support for this now exists for this in our extension; we're able to call methods from non-package based classes that can depend on jar files. We're having problems with packaged files, which we unfortunately need to work.Some problems include the fact that the extension comes bundled with R3, which is very large, and that R3 cannot be made into part of a jar file, which would make our lives much simpler.

Paper Interface

  • Low fidelity prototype: The original Anoto-friendly page design debuted during the low fidelity prototype. Listening to article actions and usage during our contextual inquiry, we added features such as fastfacts and highlighting. However, many features introduced during this stage would later be scrapped, such as quoting (due to its close relation to highlighting) and fast fact selection (since the granularity of the tagging was not at the word level). We made a fair number of changes during our actual lofi testing based on user feedback. This can be viewed between LF-1 and LF-3.
    • LF-1: Originally a normal article page would look like this. Users could underline or highlight the article, quote selected portions to show up in email, or mark text as a fastfact. +
    • LF-2: The initial summary page. This was on the last page of the article. Note the minor cosmetic changes made with pen during prototyping +
    • LF-3: Modified "on the fly" version of normal article page: we found users preferred to write comments on the margin, and would do it anyway, ignoring the comments box at the bottom. For this reason, we expanded comments to the sides. We realized the quote but was redundant and useless, so we got rid of it. A user also brought up an idea called Action Items, basically entailing an action a manager could give to someone working for them that corresponding to a part of text.
  • Interactive prototype: The interactive prototype showed many improvements learned during the low fideility prototype. Most noticeably, the final message region was moved to the front of the document pack as a cover page, since we wanted to keep the page layout consistent for all the article pages. Also, the quoting option was removed, explained above. Technical issues were highlighted during this stage, such as the Anoto pen's inability to recognize regions smaller than a certain threshold. This led to buttons (such as the fast fact, highlighter, and send button) being enlarged to increase surface area in future stages.
    • We stuck with the idea of shifting the comment areas within the article to the sides to facilitate more natural annotating for the user
    • We decided to shift the email UI onto a separate cover page in order to give the user more room for summary comments and emailing information.
    • Note the quotation tooltip has also disappeared.

Image:Interactive Prototype CoverPage.pdf

Image:Interactive Prototype FirstPage.pdf

  • Pilot Usability Study: we could not get batch processing to work and at this stage and as a result suffered somewhat in productivity. Without batch it was impossible to test other units as a coherent group. However, this stage provided useful insight which was utilized during our final design development. For example, the fast fact button was rarely used - an observation which led to the feature being eventually removed. At the same time, users continued to write in the article region, despite our specified comment region. Therefore, we created a new button which replaced fast facts and provided a normal pen palette. By default, the pen functionality worked without modification initially in the article area. However, the highlighter mode and its unknown state while using the pen motivated us to add the button.
    • The vast majority of work in this stage was spent trying to implement batch processing and decipher the XML output we were given from the pen. This eventually proved a lost cause.
    • The email portion of our project was implemented-that is, sending a group of jpegs to email addresses was coded.
    • Began preliminary work into OCR
    • Few real UI changes occurred between the interactive prototype and this stage, so no redundant pictures are included.

Most helpful evaluation technique

The most helpful evaluation technique for us was the Contextual Inquiry. We realized after the CI that our project was not headed in the right direction and would not be too useful to the people we were designing it for. This was not too surprising as we had no first hand knowledge of how our users at text100 really worked and what they needed to improve the way they worked. The CI focused our project to a very narrow group of users as well. Originally our project was to be designed for anyone working at a large company that required lots of news reading. We specifically had in mind financial firms such as Merrill Lynch. At that time our project would have functioned similar to Digg or other sites where pages were "ranked" by users. This model would not have worked for a PR firm such as text100, where many people work with many different clients and what helps or is interesting to one worker would be irrelevant to another with a different set of clients. Creating Digg-like functionality for a group of only 2-8 people, which is about the number of people working on a client in a particular office, would not allow for the "critical mass" of people needed to make ideas like Digg work. The CI allowed us to see how we could respec our project and better incorporate the Anoto Pen as well. It is worth noting that LowFi prototyping brought about significant changes to our paper interface as well. Other testing stages more or less confirmed we had a winning interface, and we just needed to tweak functionality. For these reasons the CI was the most valuable evaluation technique to us.

Final Interface

Description of Design

Functionality and Use of Prototype

Firefox Extension:

Our users will typically print out a number of articles before leaving the office for a break or to go home. The Firefox extension accommodates these practices. Users start by opening a Firefox page and navigating to an article they want to add (A). By clicking the Add To Queue button on the toolbar or context menu the user is prompted (B) whether they want to add the current tab/page to the NewsToGo queue, explained above. The queue window itself (C) is accessible from the toolbar (A). From here selected articles can be removed or printed onto Anoto paper by an intricate process: the html of each article is first pulled from the URL and converted to a PDF; from here we convert the articles to JPEG's. We size the JPEG's for our Anoto paper layout, and create Anoto pages with the JPEG's. Once the Anoto article is created it prints to the default printer. However, this entire process is ideally hidden from the user (and realistically is with a little wizard of oz to bridge between the firefox extension and java application by manually invoking one java application through a single argument, the URL).

When the user has docked the pen they may view their documents in the FastFacts tab (F). Articles can be filtered using the drop down menu and the filter button. Deletion of facts is also supported with the remove selected facts button. The filter menu is populated based on what is stored in C:\Temp\fastfacts.txt.

A help section is accessible in the extension; the help covers all aspects of the extension.

In this final stage of the extension, the FastFacts tab was transformed from a Wizard of Oz feature to a fully functional feature. Additionally, progress was made on printing articles from the extension, to a point where articles without Anoto regions are generated. Therefore printing is still implemented by Wizard of Oz.


Paper UI

Once a user has printed out their articles they will notice that every article, in addition to being parsed into separate pages and accompanied by two buttons and a column for comments on each page of the article, has a cover page that corresponds to the email functionality of the paper interface. From this cover page the user is given directions for using the paper UI and is given regions to input summarizing comments, email recipients, and categories to tag articles so that FastFacts can be sorted by the Firefox extension. Also, there is a large comment region that gives the annotator an opportunity to provide an overall summary of the entire article and it's significance. This is particularly useful for FastFacts, since the tags are now linked at the article level and the summary page would provide a fast way for the user to retrieve importante information. Finally, there is a "Send" button on the lower right portion of the cover page that, when clicked, signals the end of annotation and emails the article and comments to the emails specified by the user.

While reading the article, the user can make comments along side specific portions of the article or directly on the article. The user can also switch into highlighting mode to draw attention to specific portions of the article in the final email. Switching between these two modes is accomplished by tapping on the two buttons on the top right portion of every page of the article. Pressing the buttons multiple times has no different effect than pressing the button once; however, we allow this so that users unsure of their current mode can reselect it without worrying.

The final prototype of the Paper UI involved almost completely recoding it, as we changed to streaming from batch for data transfer. The end result we aimed for was not to stream data but use the prototype Ron is working on that implements batch through streaming via .eventData file. Some of the other minor changes we made:

  • Support for a "real" highlighter that has wide and semitransparent yellow ink implemented
  • The pen tip for normal comments was widened so that written comments appeared less pixelated and were easier to read.

Below is what a sample email that is sent out may look like. The user has highlighted and commented this article thoroughly and given a brief summary.

The printouts that the user will work with are below. The Cover Page, much like a fax cover page, appears before any page and has summary data, as well as FastFacts tags and Outlook recipients. The rest of the pages (represented by Final_Firstpage.pdf) contain a region over the article itself and a comment region for highlighting and commenting. The comment column appears on the side of the article so that users can better reference the article in the comments column.

Image:Final Coverpage.pdf

Image:Final Firstpage.pdf

Implementation

Firefox Extension:

How it was made: This was written in a combination of Java, Javascript, and XUL. This alone was a challenge as our group had no XUL experience, and the person who ended up writing the extension had no javascript exeperience. Each window is written in XUL, as is any overlay onto the browser. The functionality of the buttons is done with more or less simple javascript interacting with XUL. NewsToGo articles are stored across sessions using the Preferences functionality of Firefox. Fastfacts articles are stored locally in a file that is read in Java. Fastfacts are not stored within the extension itself because they come directly from the pen, and it was easier to write the FastFacts to a file than hack Firefox through Java. The fastfacts file is stored in C:\Temp\fastfacts.txt. The file itself consists of semicolon-separated entries on the same line; even numbered entries are FastFact and odd numbered entries are the category the FastFact falls under-there is one category per FastFact.

Hardest part: For the Firefox extension itself, the most difficult parts to implement were those that we never able to fully implement. Using Java to print Anoto ready articles from Firefox was definitely the hardest thing to implement in this part of the project. Calling a simple class worked with a framework we found online, but bringing R3 into the picture broke everything-other Java classes just couldn't see R3 for some reason.

Bridging the extension and the Paper UI(?)

How it was made: We searched online for pre-written Java code that could take in a URL an output an arbitrary number of JPEG's of the associated site. We made a few small modifications, such as customizing the image size to take up half a page, or integrating the entire code into our existing project, but the code is largely unchanged.

Hardest part: The hardest part of this section was actually finding (free) code that had this functionality.

Paper UI

How it was made: We have a few regions set up for comments, underlining, emailing, etc. We used R3's code for streaming to implement this section. Initially we were going to use batch processing, but we could not get anywhere with that. Eventually we talked with Ron and realized he was working on a batch method that already used the code for streaming functionality. We ended up deleting a lot of our original code upon switching to streaming because it had already been implemented in streaming and we had basically been trying to reinvent the wheel.

Hardest part: The hardest part undoubtedly was getting batch processing to work on our own. We had no idea what a lot of the XML was, and when we did get it working it would randomly overlap regions or not show up at all. Ron eventually explained a lot of this to us, but we switched to streaming anyway.

Email and OCR

How it was made: We used Java's email libraries for emailing, and used R3 in conjunction with the Tablet PC SDK that we got from Ron for text recognition. OCR reads email addresses and FastFacts from the appropriate region. After the computer receives data from the pen, the strokes are superimposed on jpegs of the article and resaved as jpeg's. We then use Java's built in email libraries, as mentioned above, to send the group of jpeg's to however many addresses were written down in the article and recognized by OCR.

Hardest part: We spent a long time getting Anoto strokes to superimpose on top of images of articles-colors would be inverted, things had to be resized, etc. OCR was also somewhat frustrating as it only worked on Windows XP SP2 and required streaming to work, limiting the machines it could be tested on.

Not Implemented

Firefox Extension

  • Left out:
    • Dual color rows in queue: we couldn't find a simple way in XUL to get dual colored rows as you may see in an application such as iTunes. This was an idea from our LowFi prototype.
    • Checkboxes for selecting articles in queue: the only objects we found for organizing data in rows in XUL, listbox and richlistbox, either didn't support checkboxes or allowed the user to select a row by clicking anywhere on it, not just the checkbox. Both of these were deemed unacceptable and the simple "click-anywhere-on-the-row-to-select" was kept.
    • Dates and previews inline with article title in each row: this was an idea from our lowfi prototype. This primarily was not implemented due to a lack of consistency in metadata tags for articles. Without metatag consistency across different websites, its hard to parse a page and get the information such as the date of the article, the author, or even the first few words of the article (adding to the problem is the fact that the date, author, and other fields sometimes aren't even listed!).Even with a series of conditionals to check for dates and article beginnings it is not guaranteed that the correct information will be found.
    • Having the FastFacts window only show company names, and being able to double click the company name to pull up all the FastFacts on the company. This was not implemented as XUL did not seem to support anything like it.
  • Wizard of Oz:
    • Printing. We attempted to implement this but could not finish. As mentioned previously, the extension can create JPEG's of articles but cannot "see" R3 for some reason and thus cannot create Anoto-ready printouts.

Paper UI

  • Left out:
    • Action Items, which are "actions" that one can associate with a given piece of text. For example, one might circle the author's name and have an action item be to add the author to the firm's database. As this was suggested and advocated by primarily one user, it took a back seat to other parts of our project that had a much higher priority. The lack of active interest by other users in similar functionality led us to end up not implementing it.
    • Printing/annotating multiple articles at once. The root of this problem was that we could not print articles from Firefox; had we gotten this working we perhaps would have looked into being able to annotate multiple printed articles at once, and emailing each separately when the pen was docked.
  • Wizard of Oz: we do not have any Wizard of Oz components for the Paper UI

Known Issues

Our project works as described above; however, there are several known issues involving the paper UI:

  • The time to print out an entire article with a cover page can be excessively long; on the order of about one half hour per page
  • If the Anoto dot pattern is of poor quality, text recognition (OCR) will perform poorly
  • Text recognition is poor with non-words, such as a majority of e-mail addresses
  • On the machine we were working on, the timing between the completion of Paper UI generation and the start of debugging GUI appearance resulted in very non-deterministic behavior with regards to the images of the article. Seemingly random pages of an article, each a .JPEG file, would fail to appear or render on the GUI. This resulted in completely black .JPEG files or no picture showing up on the GUI at all. This effect would carry through even in the resulting e-mail, if one was sent.
  • The 'tool' buttons were enlarged, but their images may affect the pen's performance. Registering a click on a tool is sometimes difficult.

Source Code

Firefox Extension

Image:NewsToGo FinalExtension.zip

To install the extension into Firefox, change the file extension from .zip to .xpi. A readme is included in the zipped file. R3 is excluded from the extension due to its size, as well as the fact as we don't know where to put it in the extension so that it is recognized by other Java code in the extension.

NewsToGo Code Base

Image:GGiD Final.zip

Includes code to generate articles, stream data, and email. Each one of the three folders included within should be imported into Eclipse as a standalone Java project. The GGiD project depends on the other two as well as the Anoto SDK, the R3 toolkit, and the TabletPC SDK.

Oral Presentation

The PowerPoint slides used for the presentation are located here:

Image:GGiD FinalPresentation.zip

Poster

The PowerPoint slide used to create the poster will be posted here after the Poster Session.

Image:GGiD PosterSession.zip



[add comment]