GroupBrainstorm-Group:The Anototators

From CS160 User Interfaces Fa06

Contents 1 Group Brainstorm 2 AMIA: Anoto Medical Image Annotator 3 Why we chose this one 4 Target User Group 5 Problem Description 6 Problem Context 7 Why the Anoto System is a Good Technology for the Problem 8 Solution Sketch 9 Bibliography

Group Brainstorm

1. grocery list : write desired grocery items on Anoto paper to allow ultimate list to be accessible on the Web
2. map/navigation : mark destination(s) on map to get instant navigation directions on computer display
3. circuit diagrams : Draw out schematics, computer automatically creates diagrams
4. network topology : open/close network ports, build network links just by drawing lines between hosts on Anoto paper
5. auto-webcast (notes). Lecturer’s hand-written notes are broadcast to students at home
6. photo metadata : Anoto pattern on back of photos to allow annotations / metadata to be stored
7. sheet music playback : scroll Anoto pen tip under staff to play measures of music in realtime
8. sheet music transcription/composition : hear what you compose played back in realtime
9. alarm/calendar : write down time to set alarm
10. timer : Write down a length of time, and then the pen vibrates when it runs down to 0.
11. [WINNER] surgical planning : surgeon sketches plan for a surgery, then it is automatically downloaded and attached to a patient’s file.
12. collaborative photoshop : multiple artists use Anoto paper, all mapping to same virtual canvas in photo editor (e.g. Photoshop)
13. collaborative note-taking : each writer has different canvas, but can see all other writers' canvases
14. auto illustrator : hand-drawn curves are translated to vector-based curves instead of bitmap/raster graphics
15. Anoto grading : hw printed on Anoto paper; grader comments are uploaded to a file available online.
16. live assignments : teacher can remotely watch students complete their work in real-time (on Anoto paper).
17. remote lecture feedback : students at home can submit comments/questions during lectures.
18. course assessment : assessments are automatically loaded into online course database.
19. blog/journal : blog on the go
20. sudoku : solve puzzles and have them reflect online
21. crosswords : solve puzzles collaboratively or competitively
22. penmanship trainer : get feedback instantaneously
23. word searches : solve puzzles collaboratively or competitively.
24. connect the dots : after connecting the dots, the player could be rewarded with a video on the computer.
25. kids doodle for grandparents : display it in real time.
26. recipe->shopping list : write a recipe on Anoto paper, shopping list compiled automatically
27. medication scheduling/physician feedback : the patient writes down the time/date when each does is taken, and it is made available online for the physician’s review.
28. ID / authentication : use a real signature for authentication
29. personal caterer : write food you want; caterer then brings it
30. auto autocad : autocad on Anoto paper
31. virtual/collab greeting card : friends from around the world can write signatures on Anoto paper; system then prints them all on one greeting card--authentic!
32. train schedule : circle time/place on schedule printed on Anoto paper
33. restaurant menus : user enters list of items w/ prices; system prints out menu on Anoto paper so menu is also automatic order form
34. inspection checklist : log checkoffs in real-time
35. online TANKS : paper and pen game where velocity and timing of pen strokes mater
36. universal remote : draw buttons on Anoto paper to program remote; then poke paper as one would push buttons on universal remote
37. physical therapy : writing exercises that can be remotely monitored by a physician
38. automated tattoos : user draws tattoo; robot paints it from user's drawing; can be done in real time (e.g. drawn with left hand as right arm is tattooed)
39. radiation therapy : project Anoto pattern on patient, then mark where radiation will be shot
40. plastic surgery : dot pattern projected on patient; surgeon marks places for cuts on pattern so robot can make the cuts for surgeon
41. mail order catalogs : customer writes order in order form w/ Anoto pattern
42. machine shop : project Anoto pattern on sheet metal, mark cuts w/ Anoto pen, machine makes cuts from marks
43. advertising billboard : design billboards on Anoto paper in real-time
44. instant survey : survey participants can see results instantly
45. huge survey : A poster-sized survey, hung on a wall so passersby can add their comments to it
46. scavenger hunt : pieced of Anoto paper attached to each target; players each get uniquely identifiable Anoto pen
47. handwriting->text->speech : hear what you write spoken to you as you write it
48. translator : write in language A, see instant translation in language B
49. collaborative brainstorming : rather than using an easel and poster paper, each group member could right on their respective pieces of paper and see it appear on a computer hooked up to a projector.
50. UI paper prototyping : draw paper prototypes on Anoto paper for later analysis / compilation to UI code.
51. calculator : write expression, see value on LCD screen (put LCD in pen?)
52. study drills : e.g. timed arithmetic drills on Anoto paper
53. Google searches (followed by txt msg) : write query on paper, receive results in sms txt msg--no need for computer!
54. phone calls (timed) : write phone number on Anoto paper to initiate phone call
55. text msgs Write the text message, then the phone number or person’s name, and Anoto send the text message.
56. 3D cursor/input
57. replace mouse scroll wheel : school by scratching on Anoto paper
58. pattern on sphere/other shapes : Create Anoto paper that is not simply a flat sheet of paper.
59. project pattern, use laser pointer pen
60. sketch analysis : user copies an image on Anoto pen and paper, computer analyzes user sketch to determine closeness to original image
61. texture creation : Draw a texture on the paper, then map it onto a 3D OpenGL/DirectX model.
62. scan embossed images : put thin Anoto paper on non-flat image; rub pen over to pick up displacements from bumps, then analyze to deduct physical shape of original object
63. floor planning (convention planning) : share floor plans online w/ Anoto pen and paper
64. conference scheduling : each booth has some Anoto paper; each attendee marks interesting booths as a way of getting in line (virtual queue)
65. darts : throw Anoto pen at dartboard (w/ Anoto pattern)
66. archery : attach the Anoto pen to an arrow and shoot it through a series of paper targets; accuracy is recorded from each target.

AMIA: Anoto Medical Image Annotator

Why we chose this one

We chose our surgical planning idea because it had the highest value proposition to the customer. Surgery notes are highly valuable to both the patient and the surgeon. Thus, it saves both parties much time and frustration if notes / plans were automatically logged digitally. This makes the data much more visible and accessible. We think our project will offer great value to our customer because the degree of efficacy of surgeries can be immensely life-impacting. To augment and focus our value proposition, we will design our project specifically for neurosurgeons ad neuroscientists because of their particular heavy use of medical images.

Target User Group

The primary target user group for the medical image annotation project would be neurosurgeons and neuroradiologists. With each patient, they must draw on medical images, patient history, diagnostic exams, and their personal knowledge and experience to produce a treatment plan. The same may be said for any type of surgeon, but we would focus on neurological specialists due to their reliance on medical imaging technology for diagnostics and treatment preparation. Also, we would have easy access to neuroradiologists at UCSF, due to one of the group members being a student there. In addition to interviews with radiologists, we will refer to the current literature covering computerized cardiovascular surgical planning [1-3] to investigate the critical features of medical image management. When designing a product for surgeons and physicians, it is important to remember that they need planning aids that do not impede the methods they already use with comfort. A viable product should also provide a marked advantage over traditional tools to justify the time and money spent on adapting to it. They need something that is easy, quick, robust, and secure.

Problem Description

The problem to be solved in the digital healthcare management system lies in the gap between hand-written clinical assessments and the computerized planning systems already in place. For instance, despite dramatic advances in computerized anatomical models based on medical images, cardiovascular surgeons still prefer to plan procedures using pen and paper [4]. Finite element analysis-based planning systems have received positive responses from surgeons due to their ability to predict the flow of blood and distribution of arterial pressure postoperatively [3]. However, the systems still include 2D “sketchpads” in the computer interface to maintain that aspect of the planning process. If the sketchpads are still used in some capacity, it would be logical to preserve the pen-and-paper process and find a way to integrate it with the program. The program would then be more approachable to surgeons. Additionally, the portability and easy sharing afforded by print-outs are preferred by clinicians [5]. If they could annotate the printouts and automatically add the changes to the original patient files, it would reduce the amount of time spent on record keeping and provide a more comprehensive patient history with little effort.

Problem Context

Manipulation of patient data occurs in the operating room, reading room, clinic lobby, conference rooms, and even possibly the physician’s home. Therefore, the solution to the problem outlined above would need to be lightweight and portable at a wide variety of locations. Furthermore, the ability to carry it in a briefcase would be beneficial. The solution should also be easy to learn, as surgeons and radiologists will likely be unwilling to spend much time learning a new technology that is unfamiliar and perhaps even unclear in its potential advantages. A computerized patient database would be necessary to store all pertinent data and manage the hand-written entries it receives. Additionally, any medical images would need to be stored in the same database.

Why the Anoto System is a Good Technology for the Problem

Projects designed for patient record-keeping using the Anoto digital pen have already begun [6], indicating that there is a clear interest in creating a hand-writing-to-digital interface in the clinic. Neuroradiologists would likely benefit from an Anoto-based system as they indicate possible tumors on brain scans and suggest surgical paths for resection. Anoto's critical advantage is that it maintains the familiarity of traditional pen-and-paper writing while providing digital databasing capabilities. In general, if the paper-based and computerized aspects of patient records and treatment planning can be bridged, surgeons and clinicians could ease more comfortably into the advantages offered by modern medical software packages.

Solution Sketch

Patient data forms would be printed on Anoto paper and used by clinicians to record observations and related information. To annotate or sketch on medical image, the user would select the image from the patient’s file and print it to Anoto paper. The surgeon would then use an Anoto digital pen to draw the proposed procedure in relation to the tissues and organs on the paper. Blank Anoto paper could also be used to sketch the procedure without relying on medical images. After sketching the surgical plan, it would then be automatically appended to the patient’s computer file for future reference. Other doctors could later print out the modified versions of the medical images and add their own comments. Each doctor’s modifications could be stored separately, perhaps in layers that can be view independently over the image.

The sample form seen below includes an angiography image taken from reference [3]. The patient ID is seen at the top of the form. The orange writing includes sample comments that could made by a clinician. The comments would be uploaded automatically to the patient's database folder and attached to the corresponding image, and then shared with other physicians.

Bibliography

[1] S. A. Spicer and C. A. Taylor, "Simulation-based medical planning for cardiovascular disease: visualization system foundations," Comput Aided Surg, vol. 5, pp. 82-9, 2000.

[2] B. N. Steele, M. T. Draney, J. P. Ku, and C. A. Taylor, "Internet-based system for simulation-based medical planning for cardiovascular disease," IEEE Trans Inf Technol Biomed, vol. 7, pp. 123-9, 2003.

[3] C. A. Taylor, M. T. Draney, J. P. Ku, D. Parker, B. N. Steele, K. Wang, and C. K. Zarins, "Predictive medicine: computational techniques in therapeutic decision-making," Comput Aided Surg, vol. 4, pp. 231-47, 1999.

[4] C. Taylor, "Computer Modeling and 3D Visualization for Treatment Planning." Berkeley, CA: UCSF/UC Berkeley Graduate Group in Bioengineering, 2005.

[5] E. K. Fishman, B. S. Kuszyk, D. G. Heath, L. Gao, and B. Cabral, "Surgical Planning for Liver Resection," Computer, vol. 29, pp. 64-72, 1996.

[6] P. R. Cohen and D. R. McGee, "Tangible multimodal interfaces for safety-critical applications," Commun. ACM, vol. 47, pp. 41-46, 2004.

[7] R. Kikinis, P. L. Gleason, T. M. Moriarty, M. R. Moore, E. Alexander, 3rd, P. E. Stieg, M. Matsumae, W. E. Lorensen, H. E. Cline, P. M. Black, and F. A. Jolesz, "Computer-assisted interactive three-dimensional planning for neurosurgical procedures," Neurosurgery, vol. 38, pp. 640-9; discussion 649-51, 1996.

---