Identifying Design Principles
From CS294-10 Visualization Fa08
Lecture on Oct 13, 2008
Readings
- Pictorial and verbal tools for conveying routes, Lee & Tversky (pdf)
- Rendering effective routemaps, Agrawala & Stolte (pdf)
- Identification and validation of cognitive design principles for automated generation of assembly instructions, Heiser et al. (html)
Optional Readings
- Designing effective step-by-step assembly instructions, Agrawala et al. (html)
Demonstrations
David Poll - Oct 13, 2008 02:25:38 pm
I'm having trouble running the LineDrive demo... is it just supposed to be Live Maps, or am I missing something?
Maneesh Agrawala - Oct 13, 2008 02:36:34 pm
David: The problem with the LineDrive link should be fixed now.
Kuang - Oct 13, 2008 03:24:26 pm
About Gajo's work -- the user studies to capture user preferences -- I wonder if they take into consideration the range of data. In particular we saw a UI choice posed to the user between a combobox and a slider. I think asking which one does the user prefer is naive if the underlying data is not taken into account. For instance, if the distribution were uniform vs. highly concentrated, or a high number of categorical choices vs. a low one.
Ketrina Yim - Oct 13, 2008 07:23:06 pm
After learning that a person's spatial ability can be evaluated, I was curious as to how the evaluation is done for the study described in lecture. I managed to find an example of the Vandenberg mental rotation test, but the Spatial Navigation Task proved to be more elusive.
A number of questions arose from the knowledge that spatial ability could be quantified. What is the Spatial Navigation Task? Is time a factor in scoring the mental rotation test, or just accuracy? What can a person do to improve spatial ability? Are there any other methods of testing spatial ability?
And then there was the biggest question of all: how would I fare in these tests?
Simon Tan - Oct 15, 2008 01:50:10 am
I've seen LineDrive before and it did have favorable impressions. It does a great job of clearing away the unnecessary 'chartjunk' from route maps and provides for a high data-ink ratio, just like Tufte extols.
The scaling of the roads on the map, although very necessary and intelligently done, has always struck me as worrisome. I'm sure people have used a LineDrive map and were inadvertently confused when they see a shorter line but have to stay on it for a longer distance in reality. In those situations, there doesn't seem to be a strong correlation between then map and the real world, making for a large gulf of evaluation from the visualization.
The paper mentions that an attempt to draw the maps with a sketchy rendering style, so as to imitate route maps people draw on scratch paper or cocktail napkins. This would help the perception that the roads are not to scale, but I would imagine people might still be innately confused.
Matt Gedigian - Oct 15, 2008 02:47:08 pm
The Assembly Instructions were great. At the end of the paper, it mentions that animations might allow people to deal with complex assemblies. It seems as though the hard work was already done to produce the static image instructions, and adding animations would be relatively easy. Personally, I'd love to have an animation guide me through how to assemble my furniture. Though this might sound heretical in a visualization class, I might like to have the animation accompanied by some spoken instructions. And put it all in an iPhone app. And let me call these up while I'm walking through the IKEA showroom looking at the assembled pieces. kthxbye.
Scott Murray - Oct 16, 2008 11:41:08 am
Call me old-fashioned, but I still think that even the most in-depth efforts to identify and systematize design principles sacrifices some degree of design experience that could be found in a practiced human designer. But maybe that's not really an issue; maybe algorithmically generated maps and other visualizations can be simply "good enough" for most purposes, and what we gain from their ability to rapidly scale up is worth what we lose from the personal touch of a human visualizer. The context or application domain probably helps one make that decision.
Regarding the assembly instructions, I asked Maneesh after class about more detail on how the TV stand instructions were generated. It turns out that they used only the geometric rules he discussed in the start of the lecture, but perhaps future versions could incorporate physics constraints, too. For example, the TV stand would probably wobble a great deal if I tried to assemble it right-side up; it makes more sense to build it upside down and then flip it over for the final step (attaching the top). That's something that we can sense intuitively, but for complex assemblies, the optimal orientation of the object may not be so obvious.
Razvan Carbunescu - Oct 18, 2008 12:35:42 pm
While I did enjoy the idea on LineDrive and I liked how it was able to present allot of information on a small space I believe that a map based on a reduced basemap version with 2 attached zoomed in smaller thumbnails for destination and departure points along with a list of instructions would work best for generating any sort of directions which span either a very long distance or a very short distance.
The reason I say that is because for short distances if you do take a wrong turn then you'll be able to quickly select a different road perhaps parallel to the one you were going to take initially to get to your destination.
For very long distances my reasoning would be that you would always want to stop by in some close town whenever you would feel tired or in general do something different than just following the map given exactly turn for turn for a very long period of time.
Maxwell Pretzlav - Oct 18, 2008 05:35:42 pm
I really like the idea of using a rigorous study to find good design principles. The approach presented in lecture and in Heiser et al. seemed to me almost a reverse of a typical software usability study: instead of seeing how your design works for users and revising based on feedback, have the users create a number of designs, see which works best, and refine from there. This seems like an excellent way to find new and novel approaches to design that a trained designer may not necessarily come across.
@Scott: Regarding whether automating the design process causes a loss in quality. Take a good look at the factory directions included with the TV stand. While a highly trained information designer may indeed be able to make better directions than the software presented, the reality is that most directions are not made by visualization specialists. An automated tool to create concise directions would be a boon for smaller companies without the means to hire or contract with expensive professionals.
Additionally, putting a tool like this in the hands of professionals would be an excellent combination: there's no reason a human couldn't refine and tweak the given output to better fit physical constraints and incorporate extra domain knowledge after using the software. I imagine the time and effort taken in that case would be considerably less than constructing directions by hand.
Nicholas Kong - Oct 18, 2008 06:25:08 pm
I agree with Maxwell's thoughts on putting these tools into the hands of designers. Concomitant is the question of authoring: how can we best facilitate the intuitive creation of cost functions? I think Maneesh's work on specifying label layouts by example is a great step but there are a lot of possibilities in this direction. For example, one could imagine using these automatic layout systems as seeds to an initial design, then a subsequent iterative process in which the designer could tweak the layout and recompute a cost function.
Michael So - Oct 19, 2008 07:34:18 pm
In the assembly user study reading, I found the 7 design principles agreeable. I find them agreeable because they seem to make sense, or maybe be common sense. But even though it feels like common sense, I guess they should be validated through experiments, or some type of study in order to provide some conclusive proof. I am interested in the input to automated assembly instruction design system. How does one specify the geometry of the objects? Does someone need to measure the dimensions of each piece used in the assembly process? And about specifying ordering constraints on the sequence in which the parts are assembled, how is that done? And what if the constraints inputted do not make sense such as if it is physically impossible to assemble the parts in that order. Does the planner then ignore such constraints, or does it output an error saying the ordering constraints cannot be met? This automated assembly instruction design system sounds like it might be useful for those companies that include assembly instructions with their products. It sounds like all they need is a description of the geometry for each product (and optionally labels and other constraints) to put in the system and out comes an automated assembly instruction. I wonder how much time it saves and how do companies nowadays make assembly instructions?
Witton Chou - Oct 19, 2008 07:07:05 pm
I really appreciate the LineDrive concept. Though a regular map has an accurate presentation of the planned route, it is not always useful. It usually has a lot of extraneous details, although there are times that it will help me orient and show where important waypoints are. For the most part, however, when we, as drivers, are trying to get from point A to point B, the details aren't all that important. What users want from a map is a clear representation of the key points in their route, like the important turns. And I think LineDrive does a great job depicting a 'useful' driving route rather than assuming 'accuracy' implies usability.
On the topic of animation and assembly instructions, I agree that the dimension of time will significantly improve how people approach these tasks. But furthermore, if the instructions were interactive and the user could rotate the picture to reflect how they are currently orienting the pieces, the user could potentially be even more efficient, given that the interface is designed to be intuitive enough and easy to use. It is possible that interactive instructions can be put online with Flash... although I wonder how long it will be before it is possible to include animated or interactive instructions in products that require assembly. I was picturing the moving pictures in newspapers as in the Harry Potter films when I pondered the thought of having animated instructions. That would be cool, wouldn't it? haha
Chris - Oct 19, 2008 11:57:07 pm
With respect to the reading, "Pictorial and Verbal Tools for Conveying Routes," I think that there is a really interesting research question which the authors allude to but to not answer. In particular, in section 2.2 they describe the sample directions they received as
- The resulting corpus was diverse, especially for the directions. Some were lists of turns on streets, whereas others were complete sentences with extensive descriptions of landmarks.
The question which is unasked in the discussion that follows is: "Why are there so many diverse representations?" Could it be that different people have different ways of encoding routes? Could these encodings be divided into a few distinct categories? Are these encoding categories reflective of how people decode information -- that is, can you discover what would be the most effective way of giving someone directions based on looking at how they give directions?
A variation on the experiment which they did would be the following:
- create a massive number of different environments (probably would be a good idea to do this in a VR setting)
- have the subjects do the following for a handful of environments
- spend an hour or so exploring the environment
- after the hour spent exploring, have the subject describe how to get between two locations in that environment
- find or create a metric for evaluating how successfully a subject follows a set of directions
- have each subject do the following for a handful of the other subjects' directions
- follow the subject follow the route given in the directions
- evaluate how successfully the subject followed the directions
From this data we can construct a correlation matrix of sorts of the success that subjects had following other subjects' directions. What will we find? Will it be the case that there will be a handful of clusters of individuals, where the individuals can follow each others' directions well, but cannot follow directions from other clusters? If so, what makes those clusters different? Could the subjects then be strapped into an MRI and have their brains analyzed to see if there is a physiological basis for there preferences?
Does anyone know if something to the effect the above experiment has been done? I should admit that the topic does vaguely remind me of the "learning styles" educational theory (which, incidentally, I think is largely arbitrary, over-applied, and useless).
Calvin Ardi - Oct 19, 2008 05:55:24 pm
Tversky and Lee's paper seemed to formalize colloquial and back-of-the-napkin route planning; something that we perhaps don't really give much thought to. They presented an interesting case study in which they determined that the directions and maps given based on a limited "toolkit" worked well enough, despite some of the inaccuracies found. Figure 3, for example, seemed like something that I would draw or sketch out if asked for directions or a map, perhaps with less landmarks. Their toolkit that they presented didn't seem that limited at all since I don't think most people would end up drawing intricate or very elaborate maps for directions.
I would have liked to see more content related to section 4.3, the toolkits for other domains. After spending most of the paper depicting their case study and presenting some generalizations, they move on rather briefly to a topic about different domains, and, in the previous section, a problem about translation of maps to directions (and vice versa).
It was also neat to play around with LineDrive, but sometimes some information was missing that would have been convenient. For example, some streets are only one-way; it's not explicitly there that the reverse directions are on the same LineDrive map. Also, adding another street or intersection past the destination would have helped to give some reference points. Certain locations are harder to find than others, and it would be helpful to know that point A is after some street X, but before street Y. It was interesting to do a cross country route; although not very ideal, it was interesting to see how the software would handle it (at the smallest size the map was rather cramped and confusing).
David Poll - Oct 20, 2008 05:03:21 am
I really appreciated looking at both LineDrive and the Assembly instructions. As I think I mentioned in a comment on a previous lecture, I have some skepticism about LineDrive if it's used in isolation. But, then again, that's just me. It sounds like the survey responses to LineDrive show quite conclusively that people find that visualization exceedingly useful. I still believe that it's most powerful when paired with traditional maps, since that gives you a real point of reference.
The Assembly instructions discussion was also particularly interesting. It's fascinating to me that the best man-made assembly instructions can't hold a candle to the machine-generated instructions. To me, it shows how powerful good heuristics are. While we are fairly good at seeing some of the patterns and effectiveness of certain visual representations, we struggle to take them into full account when creating visualizaitons manually. As a result, machine-generated visualizations sometimes have the advantage, since they can strictly adhere to the parameters we know are most effective for making a visualization clear.
Sarah Van Wart - Oct 20, 2008 11:32:39 AM
Just to reiterate what others have already said: as we were studying distortion in previous lectures (e.g. various cartogram depictions, fisheye zooming, etc.), it wasn't clear to me that distortion should be chosen, above other techniques, to convey a particular concept. LineDrive is certainly an exception to this. I found it very interesting that by taking advantage of some basic principles of human cognition with respect to distortion (namely preserving orientation), spatial information could be more clearly conveyed. It sort of made me think again about the Apparent Magnitude Scaling slide from the perception lecture. During this lecture, the question was asked: "When is it is appropriate to distort scaling to account for the way in which individual understands a particular visual concept (with respect to the area of a circle)?" Perhaps this relates loosely to the idea of magnifying the beginning and end points of a particular route, where the user typically pays the most attention. Though the contexts are different, both ideas distort reality in order to capitalize on human perception. The trick, I think, is to find the types of applications and scenarios where such techniques make the most sense. It was nice to take a look at some case studies that have very successfully employed some of the techniques we have been studying.
Dmason - Oct 20, 2008 01:50:51 pm
I find all the information related to assembly instructions (not to be confused with low-level programming) especially intriguing. While visiting a friend recently, we assembled a bookcase from Ikea and I remember marvelling at how direct and simple the instructions were. Moreover, I had never considered the relationship between spatial ability and assembly speed, but I suppose I shouldn't be surprised since guys frequently pride themselves on their spatial abilities in various contexts, including having a good sense of direction and building things quickly. It seems obvious, upon retrospect, that words have no place in a spatial task like assembly, or that the three-dimensionality of the object in the instruction is key to understanding how it works. Now I can marvel at Ikea one bit more, since I can appreciate what they did to make their instructions simple. No words, off-angle depictions showing depth, attention to avoiding overlap, etc.
NickDoty - Oct 20, 2008 02:10:53 pm
I wanted to follow up on Sarah's comment above and some discussion we had in class. The LineDrive demo seems like a useful example of context-specific magnification/distortion: by making roads out of scale, the algorithm effectively magnifies the smaller or more complex pieces so that the reader can see every turn clearly.
This seems much like the fisheye visualization of maps, but without the unfortunate edge effects common at the edge of a fisheye visualization. (It also has the advantages of an inset magnification, without the problems that edges don't line up properly, etc.) Is that why Maneesh likes this technique but so dislikes the fisheye effect?
Jeff Bowman - Oct 20, 2008 02:09:52 pm
I found the LineDrive paper particularly inspiring, even having seen it last year in CS160, as it makes very reasonable computational generalizations about something that previously humans could do better than computers, but that computers were more equipped to handle.
I agree with Nicholas, in that the cost functions used in the simulated annealing are what take the most time and research. I also wonder if the cost function is universal, or if it changes between times and places.
The logical extension of both the assembly paper and the Route Maps paper is to look for spatial problems that normally require human explanation, or large amounts of text. Recalling our CS160 project from last year, our interface design for transit planning—especially considering "backup plans"—requires a spatial association in conjunction with a large amount of time data.
Yuta Morimoto 20:01, 20 October 2008 (UTC)
At first, I do not think Line Drive is useful. It looks very strange and unfamiliar appearance so that I had doubted that it is really useful to drive, To confirm my intuition, I tried Line Drive to drive from Berkeley to somewhere in San Francisco last weekend. In fact, I was confused at first when I saw a shorter line but longer distance label. But, once I did not care about intuitive feeling that long distance would be drawn long line, the map was very easy. All I have to do was checking the corner where I need to turn and did not need to attention other aspects to drive. So, it may be no problem if I am accustomed to the design.
HeatherDolan - Oct 21, 2008 01:09:05 am
I was really impressed by the assembly instructions research and development. I think the study is relevant to both designers as well as those designing automated systems for design. I found it interesting that spatial abilities affected the time it took to assemble the stand, but not the instruction preferences. Like Katrina, I also was interested to know more about measuring spatial abilities and methods for improving spatial abilities.
Scott, I understand your concerns, but agree with Maxwell that an automated system is extremely useful in general and would probably increase the number of well designed and easy to use instructions. However, I do think that there will certainly be cases where a professional designer is required to resolve additional design issues and that putting the tools in the hands of professionals is the ideal solution.
James Hamlin - Oct 22, 2008 11:26:50 am
The paper on identifying design principles for and automatically generating assembly instructions was great in that it presented from start to finish a procedure for discovering effective approaches to visualization. The procedure could surely be used for many sorts of visualization, not only assembly instructions. One reservation I had, however, was that human-generated assembly instructions are only received from the students who participated in the study. It seems like design principles one might find in assembly instructions created by expert designers could be missed here. And I see no reason why such instructions could not be injected into the third stage even if no expert designers were to participate in the study. Finally, I'd just like to mention that it's interesting that the abstraction of design principles depends entirely on the authors here. They come in already, presumably, with a taxonomy of design principles that they can identify, and in principle it is possible that there are some effective distinctions among the instructions that are not captured.
Seth Horrigan - Oct 22, 2008 12:52:37 pm
When I first saw the study on assembly instructions I was quite excited - mainly because it is an area that I would never have thought of studying. I could identify aspects of instruction sets that were very difficult to follow, and ones that were easy, but I had never considered the actual process of creating these instructions. I had seen brief descriptions of Maneesh's work on creating visualizations automatically, projects which generally fall into my "cool" category within computer science work, but this was the first time I got to read and hear in depth about the process of determining what elements of static printed instructions help people the most. I was especially interested to see how many more action diagrams the "high spatial" participants created, and I was actually quite surprised to see that text did not really help much at all; although, after seeing the well-designed action diagram, I realized that the text was in fact much less efficient at communicating the process.
I have since seen this exemplified in instructions for bicycle repair and maintenance (I have had to do a good bit of this recently). The textual descriptions are very difficult to use, but well-designed images or video are hugely useful (video much more so). Perhaps a future direction would be to compare the statically generated instructions to an equivalent animated or filmed version. I would be interested to see if video or static images are more efficient.
