Task Analysis and Contextual Inquiry
From CS160 User Interfaces Fa06
Lecture on Sep 18, 2006
Readings
- Principles of Contextual Inquiry. Contextual Design. Chap 3. Beyer & Holtzblatt.
Simon Tan - Sep 12, 2006 05:33:25 pm
There's a lot of psychology going on in this article. The author has Contexual Inquiry down to an art, and I'm finding it to be a great primer before we go out and try it for ourselves. The abundance of ways to "read" a customer's expressions and behaviors is quite interesting - I'll be sure to keep them in mind if I ever catch customers looking up at the ceiling, rewording phrases I say, or say, "Umm.... could be."
I especially liked the main suggestion that the interviewer-customer relationships should be based on a master-apprentice model. We really have to let our egos go as the designers, and encourage the customer to teach us a thing or two.
One aspect the author didn't really cover was what would happen if the customer was really hard to deal with. I mean, in all industries, in all parts of the world, there will be people that we come across who would be really unwilling to cooperate. Customer service is our goal, but it seems that, ultimately, we'll only want the feedback from those customer's who are happy and willing to share their frustrations.
Ramy Ghabrial - Sep 13, 2006 04:39:56 pm
There are two potential problems I can think of with the master/apprentice relationship model.
1) The apprentice might inherit the master's bad habits. Even if this is not the case, the interviewer is by necessity locked into the customer's current work paradigm, making external ideas harder to think of. This might be remedied by interviewing multiple customers, but so long as customers work in the same field or have gotten used to existing solutions, I doubt it can be eliminated entirely. In effect, this relationship limits thinking outside the box. The interviewer can only learn what the customers can impart from the way they already work.
2) The customers may become uncomfortable, irritated, flustered, overly self-conscious or otherwise hindered by someone watching them as they work. This would affect the reliability of the data collected. The withdrawal and return method seems particularly bothersome; nobody wants to be interrupted all the time as they work.
Hiroki Terashima - Sep 13, 2006 10:08:02 pm
I don't think that we only want to get feedback from "those customer's who are happy and willing to share their frustrations" as Simon wrote; feedback from unhappy people is very important because they're dissatisfied with something about the system. I think that we should be glad that an unhappy person took the time to give us feedback.
Here are two quotes from the reading that I found useful for doing interviews: "The best cure is to pull the customer back to real experience constantly," and "a process is truly customer-centered when customers can change designers' initial understanding of the work." You want to keep the customer in mind, and be open-minded. You've got to be very patient!
David Hoffman - Sep 14, 2006 10:10:27 pm
I found that the article contradicted itself somewhat by saying initially how there were no rules, but then proceeded to list a rediculous number of "guidelines." For the most part learning about other people's problems is a skill that is very difficult to pick up from a chapter in a book. Acting like an apprentice may to some extent give the designer that he would need to design a more effective product. However, I think that there is plenty that can be learned just by watching and not having an "interview" but instead just paying attention. Probably what people say is biased or skewed anyway, so actions are probably the most important aspect to monitor.
Robert Held - Sep 15, 2006 10:15:21 am
The main emphasis of the reading was that one shouldn't try to constrain a customer interview to a rigid set of questions. They describe methods intended to make both sides of the interview equal throughout the process. They also address how to approach the "focus" of the interview. In one example of how the author provides somewhat contradictory advice, he/she states that one should have a certain purpose for the interview well-defined ahead of time to avoid wasted time on unnecessary topics. But then he/she also describes how to make the focus broad enough to go off on tangents that could provide valuable information in the end. All in all, the main point I got from the article is that it's important to let the customer control the interview topics, but not to the point where you start talking about things completely unrelated to his/her work.
Ming Huang - Sep 15, 2006 05:15:15 pm
I find this reading on effective data collection for designing a technological solution by contextual inquery quite informative. The main objective of the solution designer is to listen to customers whose lives he/she intend to help make easier. The purpose of all the techniques described in this chapter is two fold: they help the customer in remembering the difficulties during their daily jobs in concrete detail, easily, and feel less obliged to make up those details. It also helps the designer in getting the customer to give information that aids in his systematic study of the subject matter, without asserting too much of his own views and make the inquiry look like an interrogation.
A few of the above posts protested on the contradictions posed in different portions of the chapter. To me the things described in this chapter are all about human interaction and helping customers recall and refine their experiences that are relevant for the designer. The customers are ordinary people and their behavior is unpredictable. The designer should be thoughtful in their conversations. Different questions/actions/guidence may be given at differnt times that seem totally unacceptable than others. The book is trying to suggest the best plan of action during these circumstances, not a "one shoe fits all" strategy. "Be flexible" I think would be a fair theme of this chapter.
Jason Shangkuan - Sep 15, 2006 10:29:30 pm
The master/apprentice is an interesting model to describe the customer and the vendor. The four principles of contextual inquiry, however, does not give the customer enough credit by labelling them as the apprentice. The general implication of the word "apprentice" denotes someone who is new and without much knowledge but is eager to learn and follow in the footsteps of the master. In reality a customer is not someone with a blank slate and new to what they need. In some cases, the customer truly does not know, but it is more likely the customer actually understands their own needs and someone with more knowledge can help meet those needs. In addition, by the master/apprentice model, the master typically chooses the apprentice. Again, in reality, it works the other way. Since the apprentice is typically not totally ignorant, the apprentice can choose from multiple "masters" and decide which master they would work with.
Tabassum Khan - Sep 16, 2006 03:28:35 am
Contextual interviews are pretty similar to usability tests. Both are based on observation of the user's behavior. The most important part of contextual interview is to schedule it in the environment of the user not at some coffee shop or at the interviewer's office. I think that in order to make this kind of interview successful the interviewer must be familiar with the user's knowledge domain. This will help the interviewer and the user to develop a shared interpretion of the work. Contextual interviews must be performed early in the development process. This helps the designers to concentrate on the things that really matter to the users. Moreover, I believe that the experience itself should be very rewarding for the interviewers/designers because it requires them to work very closely with users during the whole project which means they have easy access to feedback.
Patti Bao - Sep 16, 2006 02:33:19 pm
I thought it was interesting how Beyer and Holtzblatt emphasized the intuitive aspect of contextual inquiry. They note that a good interviewer should be able to read a customer's stance in order to ask pointed questions to direct him/her away from abstract language, and then extract from that an interpretation that the customer can then refine. Not only that, but the interviewer is also expected to watch out for intrapersonal triggers and use "internal feelings" to guide the interview. All of these points illustrate how useful (and difficult!) contextual inquiry can be, especially because customer perceptions about their behaviour and their actual behaviour are often two different things, and none of this data could have been found through, say, a web survey.
Patrick Rodriguez - Sep 16, 2006 04:12:31 pm
I think that this article really demonstrated the usefulness of a master/apprentice model. When you think about all the different things you do, it's easy to forget the details and just go with what you know. We develop certain ways of working for reasons that we forget after some time. However, if the person asking the questions can ask the right kinds of questions, then all sorts of insight are revealed. As UI designers, we need this information to better design and refine our ideas. What does a user think of the current system that he is using? Get the scoop on what he does and doesn't like, and the way he thinks about his work, and incoroporate those ideas into a new system. Keep on iterating and both you and your customers will know exactly what is needed, and you'll be more likely to deliver.
Randy Hilarbo - Sep 16, 2006 04:24:37 pm
The concepts that Beyer & Holtzblatt presented in this article may be hard to follow but are obviously useful and seem to be the best way to gather data. I do agree about how watching a customer work can generate more useful facts than just having the customer summarize what she does to the interviewer. I think that the most challenging aspect that an interviewer must overcome is having the customer completely change the idea that he has prior to the interview. Also, stepping down from an expert role maybe a little difficult. I also think that having a list of questions (but not direclty presenting them) might help focus the interview.
Johnathan Hawley - Sep 16, 2006 05:18:21 pm
This article gives a lot of good advise on how to conduct an interview. I thought it was interesting how it played into the psychology of people in that if you take on a certain role in a relationship, the other will likely take on the complementary role. This little fact is hit upon a lot in this article in order to, for lack of a better term, ‘manipulate’ the other person into giving them the information you want. The pitfalls were insightful (interviewer/interviewee, expert/novice, guest/host). One should remember them when conducting an interview.
Scott Friedheim - Sep 16, 2006 05:32:19 pm
Majority of this paper was common sense to me. There is one idea mentioned that I do see many people suffer from and that is this: when people know a lot about a subject they feel that they can't be told anything about it because they know more than you and their knowledge is better then anything that you could possibly give them. Therefore, when trying to explain something to these "know-it-alls" you get interrupted and your sentence is finished by the "know-it-all" there by dismissing what you were going to explain. This is such an annoying breakdown in communication that I usually don't even bother explaining anything further.
The most important idea that I took away from this is something that I'm guilty of many times. When discussing Triggers he mentioned that after awhile of interacting with a person you tend to NOD in agreement because one thinks they have been there done that and they understand where you are coming from. So many times after talking with that person I'll kick myself because I realize that his prospective of the experience that I thought I had in common was in a much different context as my experience.
Sung Yi - Sep 16, 2006 05:41:47 pm
I agree with the concept of master/apprentice relationship for better interaction and effects. However, among the four principles of contextual inquiry (context, partnership, interpretation, focus), I found some possible flaws to the ideas. The idea of visiting the place where the work is being done and watch will probably bother (or make them nervous maybe) the users that they might behave differently than usual, thus possibly yielding unusual results. One other possibility of flaw is that while interacting closely with the users (partnership), the users' responses or behaviors can highly be affected by the interviewer's aspects if the interviewer is not extremely careful.
Heung Tai - Sep 16, 2006 07:57:19 pm
One point of the master/appentice model is that talking generally would omit the details. However, I think that if we focus on detail too much, we would not be able to understand the theory behind each detail action. If we don't know the big picture, it's hard to apply the skill in different situation. On the other side, I also appreciate the advantage of the master/appentice relationship. This model can help the designer to understand the practical needs of customers, this will help to avoid unnecessary assumption.
Andrew Hao - Sep 16, 2006 10:41:48 pm
The article places heavy emphasis on being in the same level as the interviewee, i.e. a partner. The goal is to have the interviewee believe he or she is sharing in a quest to distill their particular user experience. The main idea here, as far as I can observe, is to neutralize the effects of the interviewer presence as much as possible while still retaining a presence. Brilliant, but certainly it is a skill that would have to be gained through sheer experience.
One more question I have: how many samples, trials do you have to go through before knowing you're done? How many subjects do you have to get to work with you?
Suthee Chaidaroon - Sep 17, 2006 02:18:29 am
This article provides a useful method of data collection from customers. It suggests interviewers to put "less" constrain on customers, while accompany them as their partners. This article presents the master and apprentice model in which interviewers observe and learn customers' behaviors closely. I think it is quite difficult to be an effective interviewer according to this model because interviewers are required to have both communication and analytical skills. They also need to understand human psychology in order to avoid any conflict, while quickly gathering and analyze relevant data as much as possible. Visiting customers' workplaces is an effective method to interact with customers; however, real-time observation can make customers feel nervous and uncomfortable. All useful data or hidden problems might not be revealed.
Tak Wong - Sep 17, 2006 02:57:27 am
I believe the master/apprentice model is effective in creating a good design. The natural way of starting a design usually starts out with the customer giving a general idea of what he wants in, say, an application that he wants to use. Then we go program it and turns out there's a lot of specifics that we're not sure what the customer exactly wants. Usually it's hard to ask the customer right away because we don't want to bomb him with questions all the time and he might not understand our questions/needs anyway. So observing the process and asking questions at that point definitely clears up some of these confusions, but I'm sure there will still be some that needs to be cleared up.
One concern people seem to have is that some people might behave differently when there's someone observing the customer. I think if the customer does his work long enough, he is probably sure of everything that he is doing and is totally confident about it. Also, if he is interested in his work, then he will be more than happy to share every aspect of his work in detail, just like the professors talking about their research interest.
Natalie Nguyen - Sep 17, 2006 03:34:18 am
I feel that there is an obvious link to cultural anthropology here. However, when I try to put myself in the shoes of the "apprentice," I feel that there is a lot that could go wrong. For one thing, I feel that probing could get tiring rather fast for all but the most patient of customers, who is constantly asked to explain, identify, or rationalize his tasks. This would also potentially interrupt his normal workflow and thought process.
I wonder if it would be more useful to video tape the customer in his work environment and then interview the customer while watching it together. This would be much less intrusive to his normal work process, still provide context during the interview process, and would also force the customer to focus on his work habits from some distance and reflect on his practices. There are still many drawbacks in this (namely, time constraints, as well as the customer potentially forgetting details not caught on tape -- such as thoughts going through his head).
Jonathan Yen - Sep 17, 2006 10:46:31 am
The idea of contextual inquiry does not appear to be much different from traditional interviewing practices. It seems as though interviewers simply interview people on the job and are there with the customer as they do their job. Like traditional interviewing, contextual inquiry is very intrusive and full of curiousity. The key difference, I think, is that there are a lot of simplified rules that contextual interviewers need to abide by: don't take over the customer's work, try to learn from the customer, don't shift into other relationship models (i.e., master/novice, interviewer/interviewee, guest/host, etc.). Aside from all of these rules, contextual inquiry seems to be nearly the same as regular interviewing processes.
Bowen Li - Sep 17, 2006 12:11:14 pm
It seems that by making the process as "natural" as possible and have the customer do as they normally would, this type of inquiry glosses over the fact that there is a lot of room for change. While I agree that there is a lot to be learned by watching people in their natural environment, that may not be the best way of doing things. If someone has to resort to drawing artificial text boxes to keep text aligned properly, then they would probably like a better system to keep track of text boxes. However, the method of "what are you doing?" and "ok, keep doing that" doesn't allow for change. The designer has to intuit the change, instead of asking directly what would be a better alternative. And in many cases, the designer may not be able to come up with a solution as well as someone who actually uses it.
An interesting point they made is to "define the situation." The article "Anytime you want to break social norms, it's best to define the new rules for social interaction" Breaking social norms isn't usually thought of when thinking about interviewing the customer, but in this case it seems to work out well. It's interesting that if you give people an alternative, they will readily accept it as long as the rules are laid out.
Utsav Shah - Sep 17, 2006 03:52:07 pm
The concept of contextual inquiry is interesting and helpful but can be very annoying if you're the customer. I would feel bit uncomfortable if someone observes me while I'm working. People tend to stress about their work and having someone interviewing you at the same time might not help the situation. Nonetheless, it could be very exciting if you're the interviewer. I like some of the Do's and Don'ts author described towards the end, especially nodding. People tend to nod even if they don't understand what you're saying and that goes against the interviewer because the customer would naturally stop elaborating if he gets the feeling that you understand what he's saying.
Edward Karuna - Sep 17, 2006 04:50:39 pm
Some interesting points in this article involve the advocation of one specific interaction style (Apprentice-Master) above several others. (Host-Guest and Interviewer-Interviewee, among others) As I was thinking about it, Apprentice-Master seems to include several good things that can be found in the other styles, such as encouragement to ask questions, while trying not to fall into a question-answer back-and-forth devoid of volunteered information, where the interviewer/apprentice would have to hit upon the correct question in order to dredge up the knowledge that they need, which might prove impossible in some cases where the questioner doesn't know exactly what he or she needs to know. Host-Guest has a similar problem, except in the opposite direction, where the only information exchanged is information volunteered, and no questions are asked in order to entice more information out into the discussion.
Anirudh Vemprala - Sep 17, 2006 06:49:03 pm
The article on Task Analysis focused on the Master-Apprentice model of inquiry and how it benefits the design process. This process appears best suited towards making initial designs of UIs or making incremental changes to UI but doesn't seem to allow for breakthrough/out-of-the-box UI design. That being said, good task analysis has lead to great new products like Apple's Aperture (born out of discussions with pro-photographers) and improvements in applications like Final Cut Pro (born out of discussions with TV station producers/editors).
Robert Taylor - Sep 17, 2006 06:38:53 pm
This article outlines an important step in the difference between good and great design. Good design entails the designer asking questions about the users' work to get abetter idea of how to design what they are building; great design entails something more along the master-apprentice model, where the user is watched as they actively complete their tasks, bringing out the critical details of their work hiding in what was just a summary for the good designer. I also like the appreciation of interpretatoin in the text. It's like another good/great design thing, designing based on facts versus designing based on what you have gleaned and interpreted from those facts. Interpretation may atually allow a designer to pre-empt a user in what may be a good design improvement simply by critically thinking about the facts from the visit with the user.
Rayhan Lal - Sep 17, 2006 06:08:45 pm
The article is a good example of putting the end-user first. You are there to make something useful which they can use to solve a problem. Thus you must put yourself in their shoes, becoming an apprentice for your new master. It is difficult, however, to observe someone without changing their behavior. Have you ever had someone standing behind you while you typed and suddenly found yourself fumbling? It is very important to set up the right environment and to be "a certain kind of person" as expounded in the article.
Leo Chen - Sep 17, 2006 09:28:37 pm
This makes contextual inquiry slightly daunting. I always thought observing and end user would be a relatively simple task, just watch what they do naturally and build your application around trends. This guy goes so much more in depth.
However, I am not suprised to find that this all takes place in an environment that is familiar to the user. Everything here, is about observation in the end user's native environment, not in a closed study room.
I did find the master/apprentice model quite interesting, it seems very natural for people to go into this "teaching mode".
Kimberly Lau - Sep 17, 2006 09:43:34 pm
This article puts in perspective how intensive the process for understanding user needs and how important it is to gain this understanding are for designing a working system. Much of the Contextual Inquiry interview style relies on great manipulation of words to draw out the exact subtleties and potential drawbacks of the customer's work. This parallels the attention needed in developing surveys; just as one survey question may be answered in two ways depending on how it is phrased each time, a customer may emphasize different things during his description of the work so the interviewer needs to know what to ask to focus the customer on the relevant aspects. The master/apprentice relationship is most effective because it follows the well-practiced and confirmed "learning by practice" style. In addition, having the worker to relate his experience to the interviewer as he works means he will have more visual and mental cues to remind him of things that otherwise may have been forgotten. The main drawback would be not being able to achieve a good partnership between interviewer/interviewee.
Alex Wallisch - Sep 17, 2006 09:40:06 pm
Beyer and Holtzblatt suggest that the designer should try to avoid coming up with any solution until after the interview. I agree that this is a bad idea. While it's tempting to try to generate a solution on the spot and solve the problem then and there, it is unlikely that the best solution is the first that comes to mind. Furthermore, while the article's argument is that the designer can't listen well when his mind is working out a solution, I believe the much more significant hazard is that the designer will get locked into a mindset that may be hard to break out of. For example, a class I took in high school involved designing a mechanical device to perform a certain challenge. The demo that the teachers gave us involved a vehicle-based design. The task was such that there would have been many possible non-vehicular solutions, but because we'd all mentally started designing as soon as we'd seen the demo, it turned out that every group in the class made some sort of vehicle, many of which didn't work.
Vahe Oughourlian - Sep 17, 2006 10:21:26 pm
What this reading points out is that the main things that tend to go wrong with a user-designer interview are all the instictual behaviors of the user doing the work and the interviewer observing the work. The other great thing this reading does is give a general framework for the interview process. In doing so, it brings up several important points; namely, keeping the interviewer on task and keeping the interviewee on task. The most difficult distinction that I found that this reading made was the point of describing an action abstractly and summerizing the events that you really want to know about as the interviewer and speaking concretely about an action, with the precise detail required by the designer. Often I find it is difficult to prod someone into specifics, not because they don't necessarily remember the event, but they are lothe to go into the details that may reveal something odd about themselves. Similarly, many users, I feel, may be somewhat difficult to prod in this direction, as in one of the examples, where a user realized the inefficiency of what they were doing simply by speaking out the process of their actions. In that case, however, it seemed that the user was happy to continue and was not embarrased by the incident. I'm not so sure that other users may be so happy to continue an interview process in the same way when they realize that their work habits may prove to be embarrassing to themselves. Also, this falls under one of the warnings for the interviewer: should the interviewer disregard what he or she feels may be an idiosyncratic behavior that the user has corrected for himself or herself, or should he or she note this as a possible behavior for other users? I say the latter.
Kangchen - Sep 17, 2006 11:09:37 pm
I think the master/apprentice model is pretty useful in conducting an interview process. As mentioned, people usually do not give you the full blown version of their work. Even if they try to, it's quite easy to forget something. By doing it, their habits are naturally revealed and can be noted for later evaluations. Some issues I thought of while reading the article was that you actually need to conduct numerous interviews following this model to arrive at a design that will work for a more general audience. Sometimes, customers just have certain habits that they have no explanations for. Secondly, the article repeatedly advocated the interviewer should be nosy. Sure this might help reveal more details but are they the right ones? You can tell the customer to not mind your presence and a constant barrage of questions and they will probably say sure. Yet, mentally, the customer might already be unconsciously changing their work habits because they feel inclined to please you.
Andrew Tran - Sep 17, 2006 11:14:43 pm
This article gives great tips on how an interviewer should act while studying his subject. As stated in the article, the interviewer should be more like an apprentice and treat the customer as his master, letter him do his work and show you how he does it. This article also gives some good points on what to do and what not to do, such as you should be nosy and ask as many questions as possible, never assuming anything. What you should not do is to just sit and ask questions without further going into detail or staying focus on your goal. I feel this article is some what stating some general knowledge of how to be a good interviewer. I think it is just common sense to ask as many questions as possible to really understand what the customer is doing and understanding his needs.
Vijay Rudraraju - Sep 17, 2006 11:08:17 pm
This chapter gets to the heart of something that I always have found limiting about traditional interviews. The reason why television interviews are usually so horrible is that it feels like the interviewee essentially becomes the interviewers puppet. Conversely, the points touched on help me understand why the report about IDEO was so compelling. We were primarily watching team members work, not listening people talk about their job. Everyone eventually has the experience of attempting to explain to someone what they do at work. Personally, if my job has any amount of complexity, I find it very difficult to explain what it is that "I do". I never thought about why I always find it so difficult to do so, but I feel that this chapter hits the reason why exactly. Interviewing is a very delicate art. Ironically, the authors may be a victim of this exact problem because it is very difficult to lay out guidlines that someone can follow to obtain a good interview.
Yimin Yao - Sep 17, 2006 10:00:42 pm
I think the Master/Apprentice Model is a reasonably effective way for collecting data GIVEN that the apprentice is willing to cooperate with the interviewer. After reading the lengthy article, I think two points from the model that I personally found somewhat useful or interesting. First is the note on focus: a clearly defined focus will steer the interview conversation and reveals details in area covered by the focus while proventing the discovery of problems or ideas not covered by the focus. Second is the "nods": people always tend to nod when they hear opinions and thoughts that resembled their own; the article suggests that you should never assume their experience is the same as yours, or unintentional nodding might lead them to render important details unnecessry.
But how likely would the apprentice being mostly cooperative? Based on the few sections of interview scripts provided in the article, I believe that I would find the overly simple, seemingly obvious questions rather annoying and stupid (regardless of whether they actually are or not). I think that'll be the biggest barrier for the success of such model.
Udam Saini - Sep 17, 2006 10:08:54 pm
This article was quite an interesting read. Reading the article made me think of my experiences while being an intern over the summer. The article delved deep into understanding why an apprentice/master relationship is a good beginning to understanding the customer's work experience. I felt that asking how this potential design would work for the customer is a great idea in making sure that you understand the needs for the customer. Even if the customer whole-heartedly agrees with your approach, you may not use this design as asking these questions are more useful in making sure you have identified a potential problem with the current work environment. Once, potential problems have been found, only then can you really try to figure out a good design for improving someone's work expereince.
Michael Moeng - Sep 17, 2006 11:21:35 pm
I thought the part where the author mentions paying more attention when the customer does something unexpected--that is something that you assumed he/she would not do. Another aspect of the process described that I thought was important was carefully following all the steps that the customer made when performing a task (the example with sending a problem to the claims department). I found the relationship models described somewhat odd, however. While there was the master/apprentice or the host/guest models where the customer is "superior", there was also the expert-novice model, where the designer is in charge...
Huangnankun - Sep 18, 2006 12:07:54 am
The article starts off by stating that its important for the system designers to stick to the principles of having “natural human way of doing things”. However, the “natural human” way is not well defined in many cases, especially when it comes to computer program interfaces. In these cases, we rely much more on conventions rather than the “natural human way”.
The article then goes about using the apprentice model which allow designers to understand the workflow better and see all the details in its full glory. The author also mentioned that such a model will reveal inefficiencies. However, even if the inefficiency is detected, it may not be possible or even desirable to correct it. This is because most of the time, the workflow becomes so much a part of the workers that changing it will disrupt their work
The article also talks about revealing the underlying strategy of the workflow via multiple observations. However many workers have their own unique habit and by taking the lowest common denominator, the design of the system might take away from the “natural human” way of doing things for a lot of the workers. So I think its important to build customizability into the design of the system.
Another point is that sometimes, the mere effect from having an observer look at the workflow of the worker can affect the workflow of the user. Consider when having an observer, the worker might do things less casually and can be tenser. These effects can be mitigated by installing surveillance system such as an array of cameras instead of having a human observer.
I also feel the article focus too much on the detail of observation and interviewing and less on the “big picture”. It talks about the fact that talking to the management people introduces a layer of unwanted “abstraction” into the picture. However I think talking to management is still very important because they are the people who have the big picture in mind and talking to them can also unveil fundamental inefficiencies in the flow of the system.
The article then talks about how to conduct on-site interview through contextual inquiry. Its assumed by placing the observer into the same working environment and “context” as the actual workers, the observer can understand the workflow better. However the author neglects the fact that often its not the environment which gives the work its “context” but rather the training and experience which the worker has undergone that defines the “context”. This is especially true for jobs where large amounts of prior training is involved. In such cases, the observer will never experience the same thing as the worker, even if the environment is similar.
Tony Yu Tung Lai - Sep 18, 2006 12:04:14 am
Unlike the other articles from previous readings, this chapter from Beyer & Holtzblatt's book concentrate a lot more on the details of the interaction between customer and designer. The thing that I especially like about this chapter are the small details that they suggested the designer should look out for, such as the different ways for the customer to say no, and the designer's respond to the customer's comment (whether to nod or not). Although these may seem to be very small gestures, as Beyer & Holtzblatt pointed out, it affects the accuracy of the data and the customer's mindset when answer the questions.
Roland Carlos - Sep 18, 2006 12:12:01 am
Contextual Inquiry seems almost like a "duh, why wouldn't you" process at first. Who wouldn't know the product you want to improve better than the customers who use it? But the main benefit of this article is to highlight the right and wrong ways to do the interview process. My main interest in the article lied in the section about "wrong" relationship models. It warned about how the "traditional" interview process would lead to poor results, but also to avoid falling into other incorrect roles such as the "expert" role (such a role prohbits the designer from actually learning from the customer) or the "guest" role (prohibits the designer from asking the right questions).
Since the interview process proposed in the article is slightly different from the norm (based off the apprentice/master format), it is careful to establish the ground rules and rationale for doing it. It also makes it clear that you establish the ground rules with the interviewee so that they know what to expect. In fact, a lot of the interview process depends on interviewee interaction/interruption. Many of the things that you want to know about have probably become natural habits to the user, so they are like to gloss over the important details. A lot of the skill in the interview is in being able to know what to ask the interviewee and when.
In this case, interruption is a good thing, as long as it's justified (i.e. it comes attached with a poignant question).
Cheng-Lun Yang - Sep 18, 2006 12:46:14 am
From this reading we know that interviews should not have a specific format. With a specific format set, interviewer restricts the possible answers the interviewee can have. It also limits the possible feedback on different aspect of the issue. Also, the apprentice-master relationship is very true in everyday life. People often find it hard to teach by talking about everything step by step. It is easier to have some one stand by you and watch what you do. That's why there are intern positions out there. It's for you to learn while watching what the master is doing. Teaching by example is the best way to demonstrate the knowledge.
Melissa Jiang- Sep 18, 2006 12:49:16 am
I think the master/apprentice metaphor does work quite well in regards to learning about a customer's needs. A lot of the times, a customer/user does not conciously remember their daily routine because it has been so ingrained in their minds that they do not need to think about it. I remember once hearing a story about a professor and his locker. The professor had use the same locker and same locker combination for 20 years that the combination has been ingrained in his mind. One day, someone else asked him for his combination and he froze, unable to recall what the numbers were. Even after trying to motion it out, he just cannot remmeber the combination and resorted to asking the department for the combination again. Those daily routines are very important and those daily routines can make or break whatever a new designed might have in mind. Taking the apprentice approach with watching and learning their routines will probably provide more information than if simply asking the customer what they want or need.
Another note, the article states that different interviewers each have different observations. An obvious solution might just be to have a few interviewers write down their observations and then combine them later on. Although, I guess I would mind have a group of people peering over at what I do.
Maksim Lirov - Sep 18, 2006 01:11:47 am
Contextual Inquiry presents some good guidelines for interviewing the customer. It is important as an interviewer to be able to steer customers to giving feedback helpful to the design of the system. Stepping through the customer's work is a good way to catch details that the customer might otherwise not recall.
One concern I have about applying contextual inquiry is the tendency of the interviewee to try to please the interviewer. Many people think and react in a different way when they know that they are being actively watched and observed. For example, the customer may react in different ways depending on the eye contact and other movements of the interviewer. I think that the "Avoiding Other Relationship Models" section seems to cover how to react to the customer verbally, but I think even body actions have the potential to steer customer response.
I think contextual inquiry is most useful when combined with other, more traditional ways of interviewing.
David Eitan Poll - Sep 18, 2006 01:42:19 am
I think there's something particularly interesting about the apprentice/master approach to interviewing described in the Principles of Contextual Inquiry chapter we read. By assuming the role of apprentice, the interviewer places himself in a position apt for interviewing. Not only is he now in an environment perfect for observing and asking questions to ascertain data about the structure of the work the customer does, but he also puts the customer in a position of wanting to share. Instead of being interrogated, it allows the interviewer and interviewee to share a connection based upon genuine interest. I think this does a lot in terms of recalling hidden information that would otherwise go unnoticed and unappreciated. This behavioral model sets the interviewer up for success because he gets some insight into the work while building a safe relationship in which the interviewee can fully express the tasks he must complete.
Julius Cheng - Sep 18, 2006 01:22:21 am
I found this reading to be very well-structured, and a very easy-to-remember method for conducting design interviews. For beginner designers like us in CS160, I almost feel it might be worthwhile to write the four principles down on a note card and bring it to interviews. The best part about each section was the real-world examples they provided that illuminated the subtle realities of conducting interviews with customers. Details like how to know when your customers mean "no," even if they say "yes" ground the reading in possible, and even very likely real-world situations.
A good way to condense the ideal designer/customer relationship according to this reading would be "Respect and listen." Respect that the customer knows what he is doing, and listen to what he has to say about it. Again and again I hear advice like this in our reading material, and too often do we see examples of poor design that basically stemm from a failure to respect and listen.
Jae Chang - Sep 18, 2006 02:21:42 am
The main idea of this article is that the close interaction between the designer and the customer is very important. This article also points out that the designer should actually go where the work is done and observe the work process done by the customer. Although this might be helpful for the designer, this can possibly distract the customer from normality, despite the presence of the user-familiar environment. Another thing I want to mention is that this close interaction with the customer might not always work out because it always depends on the customer’s personality or preference for cooperation; someone might take active side, and someone might passive side and so forth. Moreover, the interview format is generally stated in the article; however, without specific question sets and formats and structures, it will be difficult for the designers to sequentially follow up with the customer’s responses. Lastly, about he author’s mentioning about why designers should not nod while listening to user’s responses, I think it will sometimes help the users to be confident in their responses, thus coming up with more progressive answers additionally.
CarrellKillebrew - Sep 18, 2006 02:26:53 am
Of the four principles of contextual inquiry, I found Partnership to be the most interesting & insightful point. Considering that the interaction between designer & customer is how the designer gets all the details, she must be intelligent about how she observes and probes the customer. If she doesn't probe enough then she misses important details. If she probes too much and asks too many questions, she may get bogged down in certain details, missing information about other tasks by spending too much time on one. I really liked how you got the customer involved in analyzing the structure of his work. Who better to analyze it than him, plus, it's less work for the interviewer :)
Antonis Mannaris - Sep 18, 2006 03:35:15 am
I really agree with the article's suggestion to "watch" the work in order to learn it. Most of the time the person who comes up with a great solution to a problem is one who faces the problem to begin with. Understanding the flow of a customer's routine is very important in a good design, and it is better to see that flow (context) than hear about it. Equally important is the idea of concrete data. Especially in a field like computer science where theories dominate, designing a real system is often difficult. The article also talks about partnership with a customer. I would say that we need to go even further than that, especially in the beginning. My idea of a good interview would be for the customer to describe his problems before I even make a proposal. This way, the interviewer will not affect what the customer's priorities are and what he really wants.
Sean Carr - Sep 18, 2006 07:43:54 am
The article made many good points. The use of the apprenticeship model seems like a really good one and I'm surprised I hadn't heard more of it before. When possible this seems like a great way to gather information. However, sometimes the designer doesn't have so much access to the customer. It may not be their own fault, it may be the fault of the companies they are working for, but the apprentice relationship is not always available. Also, some customers may take some of the questions the wrong way and feel insulted by the interviewer questioning how they do their job. The interviewer has to walk a fine line when they say "And why do you do that task in that way?" This question is totally valid for gathering data but may make the customer think that the interviewer thinks he is dumb for doing the task in a certain way. Once the customer feels insults, backtracking and moving forward again is very difficult.
Qingyun Tang - Sep 18, 2006 09:26:29 am
There is quite interesting relation model in this article. In the master/apprentice model, it demonstrates that people are easily losing track of why they are doing things when they just go for what they know and apply to the tasks. This could be a really bad habit for UI designers, because we should listen to customers’ request instead of making the decisions by our common sense. For data collection, I think the article does not really make it work right. As Ramy mentioned above, “the customers may become uncomfortable, irritated, flustered, overly self-conscious or otherwise hindered by someone watching them as they work.” This is a biggest mistake you could ever make during data collection. The article did not give a solution to it. People who are watched closely by others when working do not perform what exactly they do when they are not watched.
Bryant Yu - Sep 18, 2006 09:20:40 am
The problem with the master and apprentice model, or learning by doing, is you'll miss a lot of the concepts and techniques that can be applied later. Furthermore, some of the the reasoning and purpose passed down through the master apprentice model is lost. Within the context of UI in mind, this can overlook possible new ways of doing things and not building new innovations,but rather blindly following an old model that may or may not be optimal.
Another skill for a good interview CI drescribed is able to get the whole story when the customer gives u 1/2. I think it's a much deeper subject with things that a person can not be taught. A lot is senseing or having experience that there is more and molding questions to produce an effect where the customer will recall more useful information.
It was funny to see that withdrawal and return was mentioned because we had mentioned it earlier as a technique earlier in the year.
Eric Yoon - Sep 18, 2006 09:52:43 am
Many of the notions of the article were pretty commonsensical. Perhaps one of the more useful parts of the article was just the overall framing of the relationship -- looking at it as a master/apprentice model, as opposed to a interviewer/interviewee, expert/novice or guest/host model. Keeping those frameworks in mind could be quite helpful, because I would imagine it is so easy, even in the midst of the encounter, to drift into a different, less helpful model.
Gene - Sep 18, 2006 10:14:53 am
"But it is not natural to stop your work to think about it; the apprentice relationship provides the opportunity to do so." suggests that the master gains more from the relationship than I had thought. (I always thought the primary value of the relationship to the master was having the apprentice do the easy work--that and the rewarding joy of teaching something useful) This means that even the most skilled craftsmen can become more aware of their work just by having an attentive apprentice around.
Dexter Lau - Sep 18, 2006 10:21:35 am
The entire point of this reading is to properly gather the customer requirements accurately and affectively. The best way to do this is to watch the customer do work in his/her natural environment. As you watch, you must both know when to merely observe and probe further. This way you can see what is actually of importance, removing subjective skewness that may occur through conversation. Be sure to iterate through your protyping so that the customer has an active role in shaping what they really need. Be sure to focus your relationship with your customer so that you don’t stray off topic. Always try to challenge your assumptions rather than validating them, and always get feedback from your customer.
Chen Chang - Sep 18, 2006 10:36:07 am
Through this reading, I constantly get reminded that the interactional relationship between designer and customer is a delicate one. One quote that comes to mind is "Having a good ear will never get you in trouble" and this reading never forgot to iterate the importance of listening. I think the designer should always respect and listen to the customer's viewpoints as thats the most direct form of feedback to evaluate a design. Having a clear focus and staying on track to steer the conversation in the right direction and then having patience to deal with the customer can yield desired results.
Tom McClure - Sep 18, 2006 10:42:02 am
Contextual Inquiry
This scientific approach to requirement gathering is so obviously the correct way to go about design, and yet so tempting to avoid or skip. In reading this article I couldn't help but feel a little guilty about the inferior path I have personally followed in the design phases of some of my own projects. It's so easy to fall into the arrogant mode of thinking that the user knows nothing about software design or programming and therefore has nothing to contribute to the design. This conveniently masks the real fear, of course, of actually having to interact with the user. Let's face it, engineers are typically not the most adept at social interaction with non-engineers. Now that I have these principles to guide me, I think I will be much less apprehensive about committing to a session with a target user and "doing the right thing" when it comes to gathering requirements.
Bryce Lee - Sep 18, 2006 10:49:33 am
The ability of an interview to capture the natural process of a task is a difficult task, as the presence of the interviewer alone causes a disruption. There is the mindset change in the interviewee when he/she has to communicate with another individual about his/her work, causing social effects to come into play. However, this article presents a good range of way to cope with this issue, for example extracting knowledge from current action, rather than recounting. Also, I found the suggestions for the interviewer to be useful, as instinct causes individuals to look for conformation with personal assumptions rather than exploring other possibilities.
Michael Udaltsov - Sep 18, 2006 12:05:50 pm
I think it's a great idea to use apprenticeship as a way for designers/interviewers to learn about customers' procedures and work habits. From my experience with web/print design, what often happens is the opposite - designers may try to teach the customer, and justify or explain their design choices, while not understanding what the customer truly needs. The example of extra knowledge distracting from the interview also applies - instead of listening to the customer, designers may start thinking about specifics of the design, and miss or forget important details that were discussed. As a result, the final product won't be what the customer wants. Learning and understanding how the customer works may provide valuable insight into how to best interact with the customer. Paying attention to the details, participating in the discussion, and sharing potential design ideas gives more information to use in the final product, and shows genuine interest and designer's involvement with the customer.
Joe Hart - Sep 18, 2006 11:34:55 am
A possible confound with the Contextual Inquiry process is that the interviewer is in the customers face asking questions, asking the customer to go back over things, proposing solutions on the spot, etc. This seems that it would work to slow down the customer and make them pay more attention to their actions thus possibly changing their normal workflow as they try to be more deliberate about their actions. It seems that a different paradigm might work better, like video taping customer performance and analyzing it with them later asking questions. This is a good interviewing idea if nothing else is available, or when designing a system from scratch that the customer has no experience with already.
Yen Pai - Sep 18, 2006 12:17:18 pm
The advice that struck the chord with me was the emphasis on the concrete and the warning to resist the designer's natural urge to abstract. The temptation to generalize often and generalize frequently is an inclinition that is natural to any product designer and even the subject of the interview, once informed of the purpose of a designer's interview, will want to participate in the abstraction process. The article emphasizes the importance of controlling this desire and to always check ideas against the concrete reality of the interview context. One thing that the article really makes plain is that data gathering during the contextual inquiry stage is very important and a project's best chance to get accurate data. The time should not be wasted.
Jason Lee- Sep 18, 2006 12:08:04 pm
Overall, I thought this was a pretty good reading in guiding the inexperienced through the contextual interview process, as well as pointing out some common pitfalls that many are quick to fall prey to. What I found interesting was the parallel between the article and the "summary vs. ongoing experience" aspect of Context. To many, the interview process is kind of intuitive and most people have developed their own nuances in getting the material they need out of the interviewee. However, if asked to break down the interview process step by step, most people would simply answer with something along the lines of "I just kinda ask a bunch of questions...". In reading this article, it becomes obvious that there is a lot more to interviewing than just asking a few questions. Overall, I found the Master/Questioning-Apprentice model to be a good way to think about how to approach asking questions when trying to gather data about customers/users.
Siyan Wang - Sep 18, 2006 12:31:17 pm
This article seems to have a very psychological bent to it, as it analyzes the roles people fall into and their natural responses. It does, however, give some very good advice that an interviewer would probably usually overlook, such as falling into the traditional interviewer roles. Also, the reminder to stay concrete seems pretty important, like something that I would usually overlook, especially since while interviewing I would be prone to think about improvements to my system, and thus start abstracting.
Michael Mai - Sep 18, 2006 12:20:39 pm
This article makes a very strong case for using the master/apprentice model in gathering data and information from the workplace. Among the themes discussed, the aspect of partnership caught my attention. I feel that partnership is taken for granted or not taken into account in many interviews. This is due to the feel that you can't mess up or do something wrong. It places the interviewee in a position of defense and they are very cautious. By using the articles methods for partnership, the interviewee can become comfortable, leading to a more fluid and dynamic exchange of information.
Aleksandr (Sasha) Ashpis - Sep 18, 2006 12:46:42 pm
The master/apprentice model is definitely a good approach to collecting relavent data and feedback from an intended or actual user group. But when using this approach one has to be careful not to lead the apprentice too much and getting him/her to conclusions you desire instead of what is really there. The four principles mentioned are definitely the way to go, the only thing I would add is that doing the process just once is not enough, going back to other readings we have read before, a need for doing it again and again should be highly stressed.
Jonathan Chang - Sep 18, 2006 12:40:02 pm
It's interesting that, as designers, we almost need to trick customers into telling us exactly what it is they want and need from an application. Some of it's from a human need to hide inadequacies, some of it's just how our convoluted minds process and perform tasks we do so often that it's become reflex, rather than thought. How do you explain someone how to breathe? This article highlights how important it is to really get into a customer's everyday routines, into their brain, in order to do a satisfactory job of fulfilling their need.
Eric Vacca - Sep 18, 2006 12:36:09 pm
I thought this article was very intesting. The introduces various interaction models between a user and a designer during the interview process. I found the apprentice/master model most interesting and yet nonintuitive. The author used this as a starting point because of the open learning while doing enironment is conducive to much of the information needed for design or redesign of an existing model. In the end the author gives a very concise and structured model for conducting Contexual Inquiry, but i feel having such strict guidelines hinders the interviewer from finding out things that may be indirectly related to the problem. I would use the interview model more as a guideline than a template.
DavidWallace - Sep 18, 2006 12:59:01 pm
It seems obvious that design testing should occur in the environment in which it will be used. Using an apprentice/master metaphor to describe this method seems a little overblown when the main point is to listen to people talking as they work in their usual surroundings.
As a method, though, I would guess that it works very effectively. Our goal as designers is to understand what's happening in the user's head as they work, and a very good way to get that information is to have the user tell us as they work.
Siu Pang Chu - Sep 18, 2006 12:31:59 pm
This article gives a useful way to do data collecting. The master apprentice model seems very interesting for me. The interviewer should act as an apprentice who trying to learn a new skill from the master. The interviewer should consider he or she don’t know any background on the master’s work, so that they would not lose any information. Observation is another very important process. Interviewers have to go to the customer’s workplace and see the works as it unfolds. In order to produce a user friendly software, fitting in the environment is a very important issue. Also, the article mention that to understand the human psychology is also important, so that the product can avoid conflicts.
Keenahn Jung - Sep 18, 2006 01:15:55 pm
I like that this article deals very much with the human side of designing interfaces. Much of the time, your interviewee will be unable to describe fully how they use an interface, and they will be unaware that they are leaving out steps. I like how the article gives specific tips on how to read your interviewee and pull more details out of them. These techniques are very useful in many aspects of life, not just designing interfaces. For example, the master-apprentice relationship also exists when teaching a class, where the teacher is actually the apprentice, trying to learn from the student. Often times, a student asks a question about what they are doing wrong, and simply getting them to verbalize their question and thinking process helps them greatly. If you approach the problem from the canonical "right" way, and bring these expectations with you, you prevent the student from learning on their own. Acting surprised when they do something "wrong" or nodding when they do something "right" are examples of this.
Charles Lee - Sep 24, 2006 11:37:35 pm
Contextual Inquiry 1: The Heisenberg Uncertainty Principle seems to come into play here. When being observed, it's hard to not let that affect your actions. Frequently interrupting the actor with questions may be necessary to glean information, but it may put the observee at unease, which may make them fidgety and behave differently. For example, they may be more detail-oriented when they know they're under supervision, leading to a lowered importance on availability of details...or they may forget things under pressure.
Contextual Inquiry 2: I believe it is hard to confirm an interpretation on the customer's need without biasing the customer. The article's example of offering a slightly incorrect comparison in hopes that it will be righted is extremely optimistic. I think that it is more likely the worker would agree and move on, given a slightly incorrect viewpoint, stopping only to correct massive miscommunications. Bias is guaranteed anyway...it may as well come in a useful form, perhaps priming the customer towards a useful set of analogies.
Yang Wang - Sep 25, 2006 05:34:37 am
This makes a very good reading. But I have to say that most of things described in the article seems to be just pure common sense. I am not sure whether someone with good social skill will actually choose to do otherwise. But there are a couple very interesting point that I want to point out:
1. The most optimal way of relating to the customer is described as partner or even apprentice. This is a very interesting way to look at the relationship. Because in the eastern style of customer service, you always want to be as professional as possible. You always want to take the approach of expert, which is described as a pitfall int the article. In the Asian culture, acting as apprentice toward your customer simply demonstrated your incomptency, and nobody will want to do business with you again.
Yang Wang - Sep 25, 2006 05:41:34 am
2. The second point I want to point out is about interpretation. Listening is just as great part of the communication as the talking. But today, most people has lost the ability to listen and simply choose to be the talking party. The author demonstrate here how important the skill of listening and learning is. Not only we have to listen, but we have to clarify when there is something we don't understand, at the risk of looking like a fool. We also have to give customer feedback right on the spot regarding what we are interpreting. Thus, being able to listen, learn and interpret is very important, especially in communicating to the clients
Rory Martin - Sep 25, 2006 11:17:01 am
Comment 1: I like the way that the paper lays out several of the stereotypical ways that people go about conducting interviews. I never liked the interviewer/interviewee model, as it does not thoroughly cover all the material, as there are question which arise only through a more discussion based model. It is important to keep in mind that as designers we are interviewing our customers because we don't really know what they do. Once we have gained a good bit of knowledge as to what the customer does, then we can design an interface which does not copy, but rather improve upon their current methods. These interviews are for gathering information about pre-existing methods that are used by the customer, the actual design does not come into play until after this information has been gathered.
Comment 2: I like the way that this paper points out the things that you tend to avoid doing during an interview, like ignoring the idiosyncracies of the interviewee. It is very hard as an interviewer to stop yourself from ignoring things that the customer does that you think are incorrect. Even if the customer is incorrect you should take note of it in order to design your interface in such a way that the same inefficient ways of carrying out tasks does not occur in your system.
CharlesLeung - Sep 25, 2006 11:33:47 am
1) I think that the Master/Apprentice model of observing and learning is a very good one because many people are not acutely aware of exactly what they should mention. For example, if someone asked me about my morning routine, I would surely not be able to get all the details down. In fact, I rely on environment a lot of the time to remind me of what exactly I should be doing. Although it seems like common sense, the paper accurately points out that seeing the work reveals the details in a much more effective fashion than just a conventional interview.
2) Avoiding the guest/host relationship model is very important, especially for me. I notice that oftentimes when I need to learn something from someone, it turns into more of a socializing time. When that happens, I uaually don't accomplish my goals and I end up just wasting time.
Robin Franco - Dec 15, 2006 11:37:42 am
Comment 1: There was a lot of good insight in this article. What I liked most was the idea of bringing yourself down to the level of the interviewee. I know in my experience, if I know that I am interacting with someone who is much more versed in a certain subject, I am less likely to contribute to a discussion, knowing that that person could likely add more than I could. But by lowering yourself to their level, the interviewee is much more likely to open up and express ideas that they would otherwise think is stupid.
Comment 2: The idea of pretending to be the apprentice and the user being the master requires a bit of training, in my opinion. The developer of an interface has thought out his/her ideas for many days. Already gone through some sort of development cycle, so they already have formed in their minds certain assumptions and solutions. When interacting with a person, if that person rejects these seemingly well thought out solutions, the “apprentice” might be visibly offended or bothered. This would lead the user to be less candid in their response. So what would likely be best is to have someone connected to the project conduct the interview. They will be knowledgeable enough to ask pertinent questions yet have no emotional attachment to the project.
DavidWallace - Dec 15, 2006 04:13:48 pm
1). My group found that the master-apprentice model worked very well during our user sessions. Users don't seem to mind talking about their actions as they work. In fact, it's pretty clever -- people like being looked to as experts, and by casting them in the role of "masters" they become eager to share their expert thoughts with us. Furthermore, the term "master-apprentice" is a concise and easily-understandable way to explain our expectations to the user.
2). It would be more ideal if users could work alone and uninterrupted by our questions, because that is how they'll be using our products in the real world. For example, they could wear an eye-tracking device and we could analyze their session after they were done. However, since we have no way of recording their thoughts unless they tell them to us, we would be faced with the task of extrapolating their thought processes from the recorded data. In this process of extrapolation we would likely unconsciously fall prey to the error of assuming that their internal representation of the system was similar to ours.
