ContextualInquiry-Group:The Anototators

From CS160 User Interfaces Fa06

Contents 1 Contributors 2 Target Users 3 Problem and Solution Overview 4 Contextual Inquiry - Interview Descriptions 4.1 Interview 1 4.2 Interview 2 4.3 Interview 3 5 Task Analysis Questions 6 Analysis of Tasks 7 Interface Design 8 Scenarios 9 Storyboard for ROI selection 10 Storyboard for Resident Commentary 11 Storyboard for Teaching Review 12 Analysis of Approach

Contributors

Robin’s Contribution:

Participated in each interview, wrote up some of the results, answered task analysis questions, helped with easy, moderate, and hard task choices. Wrote problem statement/solution, contributed to interface design and thought up scenarios.

Gene's Awesome Contribution(s):

- Participated in first interview w/ Participant 1, took good notes.

- Edited first draft of Task Analysis writeup.

Ani’s contributions:

Participated in first interview w/ Participant 1 and took detailed notes.

Summarized key points of Participant 1’s discussion.

Created storyboards for interface.

Ed:

Wikification personified. Also wrote analysis of approach.

Everyone: Sketches and edits.

Target Users

Participant 1 Description:

Participant 1 is a Software developer for the Neuroscience lab. She develops software to prepare fMRI, PET, MRI and DTI images for statistical analysis by Neuroscience researchers.

Participant 2 Description:

Participant 2 is a 2nd-year graduate student in the UCSF-UC Berkeley Joint Graduate Group in Bioengineering. She works in a UCSF radiology/bioengineering lab and works on musculoskeletal MRI imaging. She has worked on MR in a commercial sense while employed by GE. Also, she now creates code for the region of interest-based analysis of MRI bone scans.

Participant 3 Description:

Participant 3, a Berkeley Bioengineering Professor, manages a Berkeley bioengineering lab. Reviews MR images obtained by students, uses said images in grant applications, and confers with radiologists on the development of new MR acquisition techniques.

We hope to talk to more participants going forward, in particular professionals from UCSF's radiology department.

Problem and Solution Overview

Radiologists and researchers need fast, efficient, and easy-to-use methods to share comments on images and choose regions of interest (ROI’s) for further analysis. The comment sharing process is especially needed in a teaching setting. Typically, residents make observations on images and report to a chief radiologist, who must review many sets of comments and give feedback. The solution, therefore, is to have a system where residents can check out images, add their comments, and submit them electronically to the chief radiologist. Student assessments could then be returned electronically or during a group meeting.

Meanwhile, ROI selection currently must be done at certain workstations within labs. Added portability is desired, as well as the ability to select ROI’s by hand rather than with a mouse. The solution is to perform the ROI selection directly on medical image printouts, and then have a computer automatically run the appropriate analyses when the user finishes his/her work.

Contextual Inquiry - Interview Descriptions

Interview 1

• Utilizes and modifies open source medical imaging software like MRIco and SPM for use within the lab.

• Works with a number of file formats for medical images like ECAT, NIFTY and Analyse. Each one has its own specification and developing software tools to utilize these formats is difficult.

• Medical images are stored on mutli-lab RAID device though personal DVDs have become popular for transfer of images between personnel.

• The lab focuses on studying brain changes on the elderly as well as traumatic brain injuries (TBIs).

• MRIco exports region of interest (ROI) information as a separate file that is later applied to the image file.

• ROI information can be selected manually from the brain scan image or generated functionally by doctors stimulating patients and recording subsequent scans.

• Each kind of scan (fMRI, DTI, PET) has benefits and disadvantages – its is uncommon that a patent will have multiple kinds scans on file.

• Urged us to consider NIFTY and Analyse file formats for any potential project because of its pseudo-universality in the field.

• Suggested using transparency and color information if using overlaying graphics in a visualization.

• Suggested not using handwritten annotations since reading others’ handwriting is difficult.

• Suggested a feature that would allow the images to be annotated automatically with orientation information.

• Suggested being sensitive to patient privacy when handling the images.

Interview 2

o Images obtained in musculoskeletal (MSK) group are reviewed on a biweekly basis with a UCSF radiologist.

o Radiologist reviews the images without knowing why they were obtained

o Residents go through them first, and then their comments are reviewed by the radiologist

o Radiologists typically measure vessels sizes, fetus body part sizes, etc in real time with ultrasound machines.

o Scoliosis is measured by selecting several points and calculating the angle

o Central server used for files at UCSF radiology

o PACS web-based system employed

o ImageJ is an open-source program used to open DICOM images

o In-house software used more often

o Affords ROI selection/analysis

1. ) Open file
2. ) Click “View” to see MR slices
   1. Relevant info includes pixel size
3. ) Go to a central slice
4. ) Balance image to compensate for imaging inhomogeneities
5. ) Adjust brightness/contrast
6. ) Create ROI
   1. Select Slice
   2. Select points that outline ROI
   3. Many ROI’s selected (up to 20 slices)
7. ) Run ROI analysis
   1. Pixel histogram
      1. Set thresholds for pixels to indicate which intensities represent bone
   2. Water diffusion
8. ) Press “Execute” to run one ROI or all the ROI’s
9. ) Averaging over ROI’s possible.

Interview 3

1. ) Three ways to view images in lab:
   1. MATLAB (90% of time)
   2. DISP
      1. Stanford-developed.
      2. Used for MRI
      3. Processes ROI's, statistics, etc
      4. Deals with headers of popular imaging formats
   3. OSIRIX
      1. Up-and-coming medical image processing program
      2. Most momentum is for its Mac OS implementation
      3. Takes care of all headers
      4. MATLAB-like image processing functions built-in
2. ) MATLAB image processing favored over C++ tools
   1. Programs may be slower, but development time is 4x faster in MATLAB than in C
   2. User familiarity with MATLAB is a key issue
3. ) Files maintained on a wiki on a CVS server
   1. Students deal with MATLAB-based image reconstruction from raw MRI data
   2. Assemble multiple images in .tiff format
   3. Put all of the images in a quicktime movie for easy viewing of multiple slices
   4. Movies and .tiff's uploaded to wiki for review by interviewee
   5. Accompanying emails use anatomical references to point out features
      1. This sort of mark-up is rarely used.
      2. Lab members typically discuss things in person
      3. More advanced markups for grant application figures done in OmniGraffle
4. ) Radiology markups are crucial in a teaching sense
   1. Dictation to residents
   2. Mammographers would be an excellent target user group for image mark-up tools
      1. Mammography is slowly transitioning from analog to digital systems
         1. Digital systems have the same resolution as analog system, but cost much more
         2. A key advantage for digital systems is needed to speed up the transitioning
            1. Superior markup tools would be eagerly accepted

Task Analysis Questions

1. Who is going to use the system?

Radiologists, resident radiologists (radiologists in training), radiology technicians, and medical imaging researchers.

2. What tasks do they perform now?

Observe and analyze medical images obtained using Magnetic Resonance Imaging (MRI), functional MRI, positron emission tomography, computer assisted tomography, ultrasound, and other modalities. Radiologists search the images for maladies, typically anatomical in structure. In a training setting, residents typically indicate points of interest on images, which are then reviewed by an instructing radiologist. While they also use gross anatomic images, researchers are more likely to use images that indicate metabolic activity or diffusion, such as those provided by PET, fMRI, and diffusion tensor MRI. Researchers are more concerned with the quantification of images, which involves choosing regions of interest (ROI’s). These ROI’s are use to analyze a huge range of quantities, from bone density to tissue volume, to water diffusion. The ROI’s are selected using a variety of programs, some of which are available as open-source code, while others are developed in-house for use within a specific lab.

3. What tasks are desired?

A quick, intuitive way to choose ROI’s. Our target users work with many such medical images on a regular basis, so any device or system that can improve efficiency and reduce the time spent on each image would be welcomed. If the image storage and manipulation system can be accessible from a wide variety of locales, its value to the user would also improve. Additionally, a system for the simultaneous review of resident sample analyses of images would enhance an instructing radiologist’s ability to assess their findings and return helpful comments. Another desired task a quicker, more direct way to annotate medical images in general. More accessible annotations (e.g. able to overlay on original image) are also desired.

4. How are the tasks learned?

Medical school courses and residences teach young doctors how to assess medical images. ROI processing is typically taught on an individual basis by veteran lab members to new lab mates as they join medical imaging research groups.

5. Where are the tasks performed?

Radiologists typically assess images in a dedicating reading room that includes a handful of computer stations. They also share their findings with their patients in their offices, and may verbally share findings in group meetings. Researchers process medical images at the point of acquisition, at lab workstations, at their desktop computers, and even on their laptops.

6. What is the relationship between the user and the data?

Radiologists typically order scans to be taken of patients by clinicians or radiology technicians. They then receive the images and assess them using their expertise. Researchers are more likely to be an active part of the image acquisition process, whether they are choosing a particular MRI pulse sequence or choosing a contrast agent to be administered. They are also the people who analyze the images once they are obtained.

7. What other tools does the user have?

UCSF researchers have a wealth of medical imaging software that has been developed in-house. Open-source software such as ImageJ and MRIcro are also used to open and manipulate images. ROI selection tools are typically available within the software. The UCSF software includes the ability to run analyses on the ROI’s, including pixel histograms, diffusion anisotropy measurements, and others. For instance, the pixel histogram from an ROI of trabecular bone in the femur can be used to indicate a range of pixels that represent bone. This information is then used by the program to assess what fraction of the ROI is occupied by bone tissue.

8. How do users communicate with each other?

UCSF radiology labs maintain central servers that contain all scans acquired with any of the on-site MRI machines. Berkeley fMRI studies can be stored on group RAID drives, but often researchers will store their data on their own DVD’s or portable hard-drives. When researchers want to communicate with each other about an intriguing feature on an image, they will often send an email. In the email, they will refer to a server-accessible file, or else attach an image file. The message body would then include a description of the feature to be observed, as well as the imaging slice where it can be found, if applicable. Otherwise, group meetings will typically involve PowerPoint presentations. For instance, at UCSF, all MRI scans are reviewed by a radiologist, who meets with the research groups on a biweekly basis to review his or her personal findings within the image, without knowledge of the original reason for the scan.

9. How often are the tasks performed?

The tasks are generally performed on a daily basis.

10. What are the time constraints on the tasks?

If a medical image is scanned in an emergency setting, then they are assess immediately and used to guide treatment. Less urgent scans can be analyzed and reported to the patient in a range of time, from hours to a few days. Researchers try to analyze their data as quickly as possible, but sometimes large numbers of manually-selected ROI’s are necessary, which can stretch a project out to weeks.

11. What happens when things go wrong?

In the medical setting, an error in reading an image could result in a misdiagnosis, which could potentially harm the patient greatly. In the research setting, completely unexpected results could indicate a misplaced ROI, which can be easily replaced. Care is taken to prevent the overwriting, deletion, or general loss of medical images. Thus, errors in the analysis phase are typically easy to rectify.

Analysis of Tasks

Easy Tasks:

1. Circle interesting features - here the user marking parts of a medical image by drawing a circle around the interested area.

2. Jot down freehand comments - user handwrites comments of image either on a printout of the image or on a seperate piece of paper.

Moderate Tasks:

1. Carefully draw borders around regions of interest - user precisely marks the border around a specific region on interest on a medical image (e.g. a white matter tract connected to the telencephalic hemisphere of a brain).

2. Aggregating typed comments with original images - user types textual comments on a computer that corresponds to a particular image. The difficulty lies in attaching the text to the image in a useful way.

Hard Tasks:

1. Receive separate comment/drawing images from several people and individually review and respond to each of them. - this is used particularly used in the context of training, where a master radiologist train resident radiologists by having them practice analyses of images.

2. Indicating specific analyses to be performed on an area of interest (ROI) - each organization, because of their different data and tasks, tends to handle ROIs differently.

Interface Design

Based on the feedback we received from users, several design related constraintts were discovered. This pushed us towards deciding the following design goals:

1. Present the users with digital versions of medical images in a clear and simple way. If available, allow for orientation (left-right, dorsal-ventral etc) markings to be overlaid on the image to assist the medical professional. The initial implementation will be designed for use with MR image sets. The user can load a set, and then rapidly move through the slices to obtain a desired view of a certain piece of anatomy.

2. Allow each users' annotations to be stored as meta-data and not over-write the existing image file in any way. It is crucial that the original images are not modified in any way.

3. Allow the users' annotations and ROI's will be layered on top of the original image other annotations. Currently, ROI's are saved as separate files, so the new implementation will not be a large departure from previous ROI management systems.

4. Allow a master user to collect annotations from the existing users for the purposes of discussion. Permit the master user to return comments to the "student users" using the same program.

With this in mind, we came up with the following outline for an interface.

The interface we had in mind was one that resembled a multi-document image viewer with a preview pane on the side. The preview pane would allow quick selection of image sets from within a given directory. The main document space would present its user with the medical images under consideration, along with representations of layers that store the annotations made to the images. A slidebar at the bottom of the screen would allow the user to rapidly scan through the slices within the imaging set. For a given image, a set of layer thumbnails would be displayed near the bottom of the screen. Each layer would represent comments from a different user. By selecting one of the thumbnails, one could place that layer over the original image in the main window. A "Comment" button would then create a dialog box where the user can add his/her comments about the currently viewed layer. These comments would then be attached to that layer's metadata, and would be viewable by the original creator of that layer. Options to help orient the image, create text boxes over the image etc can also be placed strategically in the main document space. Additionally, a "Print" button would be included. After selecting the print button, the user would be prompted to print either an ROI printout or a commenting printout on Anoto paper. A commenting printout would only include the original image and a "Done" checkbox. An ROI printout would include the original image and "Done" box, along with checkboxes for running certain types of analysis, such as pixel histograms, once the Anoto strokes are uploaded to the computer.

By making it resemble a common multi-document image viewer, we hope to reduce the learning costs associated with this application and have it more easily integrate with the work cycle of our target users.

The benefit of adding a hierarchy to the image sets (using a "Tree Combo" like the one featured in Windows Explorer) is that it allows the users to associated a variety of image files with a given patient's digital record or associate different kinds of images for the purposes of illustrating a point (as a medical instructor).

Scenarios

1. ROI selection:

A specific slice is chosen from an MR image set. The user designates that he/she wants an “ROI SELECTION PRINTOUT.” The image is printed on Anoto paper. The researcher draws a circle around a bone in an image, then checks boxes that read “Pixel Histogram Analysis,” “Mean Pixel Intensity,” and “Area.” After the selections are finished, he/she checks the “Done” box. When the pen is returned to the USB cradle, the computer reads in the ROI’s, finds the original image, and runs the analyses.

2. Resident Commentary:

A user logs into the system with a unique ID. A specific slice is chosen from an MR image set, available from a central server. The user designates that he/she wants to run an “IMAGE REVIEW.” The image is printed on Anoto paper. The user then is free to circle anatomical features and write in comments. After the comments are finished, he/she checks the “Done” box. When the pen is returned to the USB cradle, the computer reads in the comments and uploads them to the central server with a tag for the user’s ID.

3. Teaching Review:

A chief radiologist logs onto the system. He/she then accesses an imaging set from the server. He/she can then see a list of users who have submitted comments for each image. The radiologist can select a specific user, and then that user’s comments appeared laid over the original medical image. A text box appears at the bottom of the screen, allowing the radiologist to enter his/her comments on the student’s observations. Those observations can then be accessed by the student.

Storyboard for ROI selection

Storyboard for Resident Commentary

Storyboard for Teaching Review

Analysis of Approach

This solution makes use of the special abilities of the Anoto pen through its greatly increased portability. At group discussions, each person could bring their own Anoto printout of the images to be reviewed and add their own comments throughout the course of the meeting. The ability to share one's personal comments and sketches over a network exploits the Anoto pen's digital affordances. Furthermore, annotation and selection of Regions of Interest can now be done with the much more natural pen and paper interface instead of cumbersome mouse interfaces on the computer. This also has the added benefit of creating a paper hard copy automatically.

---