Task Analysis and Contextual Inquiry

From Cs160-sp08

Jump to: navigation, search

Lecture on Feb 7, 2008

Slides

Lecture Video: Windows Media Stream Downloadable .zip

Contents

Readings

Jonathan Chow - Feb 04, 2008 04:10:32 pm

While reading Beyer and Holtzblatt's piece, I couldn't help but feel that they were a little roundabout in their descriptions of relationship models. Most of the readings that we have had so far have a very definitive stance about how they think you should act. But in this piece, it seems like Beyer and Holtzblatt name a number of different styles of relationship models, but do not suggest which ones you should use. They do name the interviewer/interviewee and expert/novice as poor models, but I am uncertain which model they prefer between the master/apprentice and partnership model. This could also come from my perception that the those two models seem very close in nature. Both involve the customer doing their regular task with questions being asked by the interviewer.

The point that I did find to be very striking in the piece was the suggestion that having an initial model to iterate with confines the possibilities to that model. I like the suggestion to make every suggestion to the customer as you go. Although, it would seem to me that you would almost need a dedicated customer on the team to go through all these steps with them.

Eric Cheung - Feb 04, 2008 07:59:21 pm

There seems to be a tricky balance between being nosy and interfering with their work to the point where you don't get any useful information. On the one hand, you need to understand what the user is doing and why they are doing it. On the other hand, you don't want to annoy the user to the point that they truncate their usual workload just to avoid having to answer more questions. The partnership section of contextual inquiry walks a fine line between the apprentice-master model and a peer-peer model. It says to "let the customer shape your understanding of the work" but it also says to "teach the customer how to see work by probing work structure", so apprentice-master seems like a bit of a misnomer.

I'd be kind of curious as how to select the right type of user for contextual inquiry for your needs. Granted, it varies from person to person, but it'd be important to make sure that what they do is close enough to what you want to observe so that you are not trying to push them into doing things just to meet your requirements.

Katy Tsai - Feb 05, 2008 12:23:43 am

I agree with Eric in that there is a very fine line between interfering too much with your user and his or her environment and probing so you get a sufficient understanding of one's actions. I think the key that Holtzbatt continually emphasizes is to be able to understand the motives behind one's actions as well as the details of the process people go throughout their daily functions.

I also was intrigued by the constant push to enter a certain context without an agenda. It's surprising to hear that we shouldn't plan our our interactions or "interviews" when our lives are constantly forced into structure and organization. What I have seen throughout these articles is that creativity isn't something that can be confined and controlled; it involves freedom of thought and an ability to observe things as they are.

Max Preston - Feb 05, 2008 01:06:11 pm

I was a little irritated by the way this article was presented. Even though this is an article about human interactions, it felt like he was describing how humans learn, interact, and process information as if we were robots or at least some sort of inhuman deterministic system when there is in fact a lot more complexity and many more individual and situational factors involved in these processes. I personally feel as though a strong intuition is far more important than the sort of simplified structures and procedures presented in the article. In practice, attempting to artificially create a specific type of relationship with another person tends to turn out horribly wrong. Such things should instead come together naturally.

Henry Su - Feb 05, 2008 02:44:14 pm

I found Beyer & Holtzblatt's article very intriguing. In some ways, I agree with their major points. For instance, it's certainly common to be in a situation where you'd like to learn something, but the person who is "teaching" you ends up summarizing, glossing over details, and skipping steps. Watching someone do the work, while asking them questions, does seem, at least theoretically, like a much better method to squeeze out the crucial details. However, I mentioned "theoretical" because I think the method will only work under certain circumstances. For instance, nearly all the customers' tasks mentioned in the article (checking accounting records, routine for starting the day off, etc) are mundane and easy. This makes the master-apprentice and partnership models effective, because the customer doesn't require much concentration. If, instead, we tried this on a customer of a code-editor (like Emacs), it may be quite a bit harder, because the customer may be involved in a deep train of thought while debugging, and having the "apprentice" there interrupting unpredictably may be unacceptable. I'm not saying that we should throw away this master-apprentice model, but rather that it's much easier said than done, and practice is needed, because not all customers have an easy time naturally doing their work while being examined by an outsider who interrupts frequently.

Michael So - Feb 05, 2008 05:30:23 pm

The master/apprentice model seems agreeable. Seeing how the customer works will reveal important useful details for the designer. Being lectured, or being given a summary is good for conveying abstract topics, but not at revealing details. Instead summarizing removes details, or glosses over them, as the article points out. Watching someone actually doing the work makes the abstract concrete. You see what the person does, how the person behaves, and the decisions the person makes. Getting the customer to talk as he or she is doing the work is also good because the dialogue will contain more details to the task the customer is doing. I think what the article suggests on the interviewer to do are good suggestions. However I am a bit unsure on how an interviewer should act around a customer who has a bad temper, or who is really stressed. I think those type of customers would be more challenging to get them to divulge important useful details for the designer.

Bo Niu - Feb 05, 2008 06:31:55 pm

The idea of going into the work place and act as apprentice seems really useful in order to extract the important information from the user, and the things to avoid were very reasonable as well, but as Beyer and Holtzblatt wrote that designer just doesn't have the time to learn the job as an apprentice. They have suggested the four principles to help solve this problem, but all these principles are trying to shorten the apprentice learning time by going back to the interview mode. I think the trade of between time and quality of the inquiry is still an issue, and where the line should be draw have to be considered with the current situation in mind.

Nir Ackner - Feb 05, 2008 07:45:25 pm

Several of the previous comments have discussed the issue of interrupting the interviewee, especially when they are doing a critical task. One technique that doesn't seem like it has been considered is videotaping the entirety of the user's work period, without interruption. Then, the user and designer could sit down to examine the whole process, using technique similar to that described in the article. This would ensure that the user behaves as they would without observer, but still provides the designer with the opportunity to probe, ask questions, and understand user strategies and needs.

Gerard Sunga - Feb 05, 2008 10:36:20 pm

An apprentice-type relationship seems useful in discovering what the true needs of the consumer are, revealing numerous details that would otherwise be left out in a typical business meeting, especially as revealed through periodic interviews regarding questions that arise during this "apprenticeship." As others have mentioned, my main issue with this process is the possible detrimental effect it may have due the influence an observer (particularly in the increased pressure brought about by the mere presence as well as the possible interference and resulting agitation as a result of being assaulted with a barrage of questions). As the previous poster stated, it's probably best to follow this process in an after-the-fact manner, retaining all the benefits of viewing the consumer in a natural work environment without the higher possibility of the observer disrupting the work flow.

Jonathan Wu Liu - Feb 05, 2008 11:11:55 pm

I think contextual inquiry may be more objective than other methods, but probably not the most objective for some people. If a random "designer" came to observe someone, I imagine they would be nervous while the designer is sitting next to him observing every move. The person might be much more careful in what he did, conscious of the fact that the designer is watching. Namely, I might go through extra steps of work to show that I was a careful person, or he might do everything more quickly than normal to show his proficiency in his work. Just the fact that an unknown person is observing a person makes it natural to suspect that the unknown/vague upper management is evaluating employees. Perhaps a better way is if a designer trained one person in each group that most employees are familiar with and have that person report to the designer.

Randy Pang - Feb 05, 2008 11:49:39 pm

Overall I felt the article was an interesting introduction to contextual inquiry. I do agree with Jonathan (Chow) in the sense that they didn't offer a clear model to follow, but I think that's fine because I don't think such a model exists. It depends on both the context of the situation and the exact moment you're observing. I think as long as you're flexible and remember your true goal (understanding what's going on), then you'll find the models manifesting and taking care of themselves. I also definitely agree with Jonathan (Wu Liu) that ones mere presence, however unobstrusive, affects the work flow to a non-trivial degree. That being said, however, this seems to be a neccesary by-product of this methodology. Overall I think the fact that you can't seem a completely natural environment is unfortunate, however, I think that this is easily made up for with the knowledge gained from proper inquiry (perhaps that secretary wasn't doing exactly what she was doing each morning, but the important thing was to notice the important details and issues, which I believe will come up regardless if they're a little nervous or not, and inquire into them). In response to Nir, I don't think that being silent and videotaping would be nearly as effective. This is mainly because, as the article states, you want to catch people right when they're in the middle of their train of thought. If you ask later, you're more likely to get abstractions.

Chris Myers - Feb 06, 2008 12:52:25 am

Most definitely. I think it is incredibly important to be in mind of the user when designing something. I would argue one step further and have the designers actually work on the job of that the software is being designed for.

For example, emacs is such a good text editor for programmers because it is written by programmers. Additionally, it has evolved over a very long period of time and millions of programmers across the world can modify it.

At junior college, I took a class called 'systems analysis and design'. Essentially the entire class was what was outlined in this paper. But I learned more from this paper than from that stupid class. All the pitfalls mentioned in the paper, yeah, we fell for them. Nowhere did we learn how to extract details from the customer by probing. Had I read this paper before taking that class, It would have turned out a bit different.

Benjamin Lau - Feb 06, 2008 01:33:15 am

Can't really say much about this reading. It's a bit dry but the heuristics offered are good. Instead of offering a bunch of rules to memorize, it instead gives the reader a single system to follow in the form of relationship models. An additional benefit is that the reader can create more (of his own) ideas for contextual inquiry by extrapolating even further from the master-apprentice and partnership analogies.

The part I liked the most was on how to overcome customer politeness. The author explains how to catch the real intention of the user and figure out what he is trying to communicate. A "yes" could in fact be a "no" if the user doesn't agree immediately. The problem of customer interpretation is discussed very thoroughly. One thing I thought needed elaboration though is how to create questions that are specific enough to guide the customer in interpreting his own behavior, but general enough to avoid leading the customer around and incidentally missing key points important to him.

Khoa Phung - Feb 06, 2008 12:34:24 pm

This reading displays the different techniques for interviewing very clear. I was not aware that different approaches can affect the result as well as the quality of the inquiry so dramatically. It is very enlightening to see how to use the master/apprentice relationship to get a more real model of the customer. I thought asking questions is only positive, yet from the text it shows that too many questions as well as bad questions can result in complete different answers and insights. Therefore, the interview as simple as it sounds will require a lot of expertise and experience to master.

JessicaFitzgerald - Feb 06, 2008 01:29:18 pm

I thought the author's interpretation of acting similarly to an apprentice was an interesting argument. In order to learn how someone does what they do, you have to watch them and see how they use their tools on a regular basis. Then by seeing how they use their tools, you then are able to improve those tools to make their life easier and more efficient. I also found it intriguing about how he was concerned with the way you ask questions and find information. You must ask in the past tense about what they did, instead of asking what tasks they perform. I find it interesting that when someone asks you to tell them about something, you summarize and give an overview of what you did, instead of going through all the details and saying what you do at work step by step. I think this article has to do with how we behave in society and natural impulses than explains how to build an effective user interface.

Glen Wong - Feb 06, 2008 01:40:22 pm

I have to agree with some of the other comments that this reading was quite a bit drier than the readings we've been getting recently. That aside, I think the advice given on how to conduct Contextual Inquiries is very nice especially for someone who does not have any experience and wants a good guideline to follow. Personally, I feel that a lot of the advice that was presented is very common sense. While the apprentice-master relationship was interesting to me, I feel that this is something I would naturally do when performing a contextual inquiry. The other ideas presented, such as being focused and trying to interpret your observations while watching your customer, were also very common sense to me. That being said, I will again reiterate that I feel this reading is useful if only as a guideline in your mind for the beginner contextual inquirer.

Yunfei Zong - Feb 06, 2008 03:43:04 pm

The reading provided a very useful way to get an understanding of how the user things. Sometimes I get asked to write code that will process large amounts of data at my workplace, and although I can understand the specs and what is required, I rarely understand what kind of extendibility that the code should be able to handle. I solved this problem by using a simplified version of what was presented in the reading; I watched a scientist using the data, and watched how he acquired the data and how he used it. It was quite easy to notice that the scientist was taking too many steps in order to get finalized data; he first had to scan in the bio-data using a specialized piece of hardware, then export the data, then run my code, then use some other programs to make the format understandable visually. By watching the scientist, we were able to build an application that integrated all these steps, and made it very simple and fast for the user.

Michelle Au - Feb 06, 2008 03:51:01 pm

Beyer & Holtzblatt bring up a good point about the importance of interpretation in contextual inquiry. The examples of indirect ways of saying "no" illustrates the point that pure observations are not good enough; the underlying meaning has to be interpreted correctly and the interviewer has to react accordingly. In the case of an indirect "no", the interviewer should follow up with an appropriate reply to get the needed data. The dynamics of the relationship between interviewer and customer also affects the customer's responses. If the relationship is not very strong, then interviewer receives less informative responses. In these cases, interpretative skills are essential to steer the customer closer to the focus.

Ilya Landa - Feb 06, 2008 05:05:13 pm

This article, although not as interactive as the last one provides many important tips to conducting a successful interview. Many people commented earlier that the given rules are too dry and automatic, and describe a utopian interview rather than a reality. However, this article feels like it was written as a guide for people with little experience doing such interviews. The authors were giving an outline of what a person unfamiliar with the process should attempt to achieve and what one should avoid. Some advices might not be perfect; the directions to be overly nosy, to ask about phone calls, and to follow a person around the office could intimidate or annoy the interviewee, along with a flood of “positive attention” that is supposed to incite partnership. But, overall, the directions felt correct and beneficial – do not intimidate an interviewee with your technical expertise, do not come with a fixed questionnaire, no dot ignore actions that seem pointless or irrelevant, etc. After reading this article, I feel that the points given in it would really come in handy if I ever have to do an interview like that.

Hannah Hu - Feb 06, 2008 05:18:59 pm

While the guidelines outlined in Beyer & Holtzblatt are indeed useful, if not critical, to understanding customer needs, concerning the interaction between interviewer and customer, I felt the guidelines given for that area is pure common sense. One could argue, however, that "common sense" would apply towards normal social interaction, but in terms of client/agent interaction, which is at best on a professional formal level, the type of interaction in Contextual Inquiry may seem somewhat surprising.

I would also like to add that the guidelines can be applied not only to client/agent interaction, but also to intra-community interaction. Perhaps you see someone sitting in a lonesome corner who could use some social interaction. She just so happens to be developing an application that you, coincidentally, have dreamed about for years. Following the advice of Beyer & Holtzblatt, you could forge a long-lasting partnership with that person. To say the least, the reading could well be "Social Interaction for Dummies".

Reid Hironaga - Feb 06, 2008 09:02:47 pm

Beyer & Holtzblatt assess some interesting points in the models taken to learn about clients in their natural environment. I think their Master-Apprentice analysis is decent, but more strict than it has to be. Since information is conveyed in different ways for different interviewers, clients, and context, the model is too strict for generality. Also, I was bothered by their mapping of the terms "Huh?", "Umm... could be", and "Yes, but..." as deterministic expressions of other expressions. I know people don't always mean the same thing even if they utter the same words. I appreciate their focus on the need for a desired level of focus that will optimize the data and interpretation relevant to the designed product, but I think that an attempt to filter data may cause more harm than allowing all data to be collected and analyzed in its entirety for later consumption.

Cole Lodge - Feb 06, 2008 11:00:36 pm

This reading, in my opinion, was very odd; it felt as if Beyer and Holtzblatt were trying to fit every customer interaction into a nice box. They acted like the customers were drones, being able to control and predict their responses; for example, if the interviewee looks up, he must be talking generally. Although these context clues can be very helpful, they should not be the end all in the interview. I also found it odd that the authors favored the interview and partnership methods over the apprenticeship method. The only downside I see to the apprenticeship method would be the amount of time it takes; if I was developing an application, I would much rather spend the time to understand the users needs by understanding their job.

Harendra Guturu - Feb 06, 2008 11:39:53 pm

The Master/Apprentice relationship model for effective data collection was also the rough model I had in mind. I felt this way since in a lot of CS classes, I learned the material much more thoroughly after doing the actual coding assignments rather than learning about the material abstractly from the lectures and readings. The process of doing the task clears up a lot of information that gets lost in interpretation when explained using words.

One of the earlier posters mentioned using videotaping to observe the user. I feel this can give a mixed set of results. It may be good in that the user will behave more naturally, since they do not have someone directly observing them, but it could also be bad since the video may not have the resolution that is required to capture all the detail that may be necessary to assess the situation completely.

Jason Wu - Feb 07, 2008 12:21:15 am

Beyer & Holtzblatt have a good set of general rules for interviews. The models, Master-Apprentice idea seems good, and focusing on during the task seems to be a good idea. While many seem to believe that they treat humans as a deterministic machine in the article, I feel that as long as one treats their content as good guidelines and heuristics, it should be fine. It is corAdaprect that not everyone will fit in Beyer & Holtzblatt's perfect model, but I believe for someone that's a bit new, it contained a lot of good concepts for customer interaction in general...as long as adaptability follows.

Jiahan Jiang - Feb 07, 2008 12:36:34 am

The Beyer & Holtzblatt was a fairly dry piece; they had some useful information that could be conveyed much more concisely. I really enjoyed the master-apprentice discussion because it shined new light on the way people communicate their work to others; the contextual interview format at the end seems good on paper, but I'm not sure if it will work out as smoothly as it sounds. Somehow I think it would be a bit awkward for people that are not used to it. Also I'm not 100% certain how "partnership" is different from master-apprentice because from the example it seems to be precisely that relationship.

Fan Yang - Feb 07, 2008 12:46:49 am

I like the idea of the apprentice/master relationship and using it to find out more about how the customer does his work, but at the same time, it feels like it could be a bit too intrusive. I know some people would get really annoyed when you ask them why they do something and then keep latching it on again with "well what's the reason for that". I understand that its necessary to ask the questions but it just feels a bit too annoying. I agree with the idea of the summarizing part, most people don't go into a lot of detail when they explain something. Its much more useful to have them explain it while they are doing it because its already in their thought process.

Benjamin Sussman - Feb 07, 2008 01:02:08 am

One thing that really struck me as true in the Beyer Holtzlatt article was the tendency of humans to summarize, and how the information conveyed in a summary is not at all what is important when it comes to improving design. This is because so much of the importance of design is in the details, and these are lost in big ideas of the summary. Take for example the iPhone, a tremendously successful device, however in summary it is nearly identical to every smart phone: it has an internet browser, QWERTY keyboard, music and video stored inside, and text messaging. However any iPhone owner will tell you in great detail the big differences if you prod them enough. If you simply asked users about their cell phones, you wouldn't learn much more than their need for the features listed previously. However spending time with them and interacting with them while they use the device would provide an investigator with details about how and why those features are used and what are the users favorite and least favorite aspects of those features.

I also related to the idea that you need to experience the user in the process of using the product in question while also interviewing them. This way you can learn the language of the product and relate that to the actions performed to use the project. A designer needs to understand both the actions and the corresponding language in order to simultaneously better the product without confusing or distracting the users with the improved features.

Daniel Gallagher - Feb 06, 2008 10:58:27 pm

The Beyer & Holtzblatt article made more explicit what I see as the 'intuitive' method for getting information out of people without them over-thinking their words. The master-apprentice model seems like a label for idealized teacher-student relationships, with the stipulation that the teacher is instructing-through-doing rather than just lecturing. Getting things back on track if there is some kind of interruption or the interviewee gets frustrated seems like the biggest problem that is partially addressed in the chapter. I could see how someone who had difficulty conducting interviews could use the information in this article to reevaluate themselves, but I also feel that most of my classmates (and other successful students) have probably attained enough understanding of the connection between acting a certain part to elicit cooperation or information from another person. Whether this is intuitive or developed, it is clear how the subtle relationship manipulation described by Beyer/Holtzblatt would come in handy in arguing grade changes, extensions, getting additional academic or non-academic help, dealing with bureaucracy and other situations a Berkeley student might come across. The chapter's lessons may in fact come in more handy for those who have forgotten the unofficial lessons of their school years, especially in the case of a public school.

I'm actually really curious whether this seems sensical along with the reading and everyone's thinking.

Scott Crawford - Feb 07, 2008 01:30:03 am

As usual, one of the main themes was to 'think outside of the box.' In this case, it pertains to the 'focus' aspect of a good contextual interview in which the interviewer has to break past any prior assumptions about the customer's work to see what's really going on and what matters. Another thing I sort of liked was how they don't explicitly discourage design ideas on the spot. My personal feeling is that a better balance might be struck if the interviewer goes in actually prepared to bounce ideas off the customer (not prior ideas, but ones that as a designer they will undoubtedly conceive during the interview), but to not spend 'too much' time dwelling on any one idea. Summarize it quickly (i.e. write it down while you describe it to the customer) and then record feedback, but don't try to optimize it or explore it further (if the idea is 'vague' and the customer feels inclined to fill in some blanks for clarity, let them do so, but don't actually ask them about anything except viability/usefulness).

Alex Choy - Feb 07, 2008 02:01:44 am

Beyer and Holtzblatt go through different things to consider when performing a contextual inquiry. They emphasize the relationship with the customer. One thing that I found was interesting was the fact that peoples' daily routines can sound mundane and brief. However, when actually performing the task or going through daily routine, people can more easily elaborate on the thing that they are doing (like in the secretary example). This is similar to the master/apprentice model where the master simply goes about his daily routine and simply elaborates on what he is doing in order to teach the apprentice. People have mentioned that it is hard to interview and understand a user while also not affecting their work. This can be important and affect the things that they say/do. The article mentions that the interviewer must be focused, which can allow him to steer the conversation. However, while having focus can reveal more details and facts, focus can also affect the customer's attention. As a result, they may forget about a small and/or minor step in their routine simply by having to multi task (answering questions, worrying about the interview, and the task at hand).

Brian Trong Tran - Feb 07, 2008 02:26:24 am

At an info session by Intuit last night, a representative said that they used to literally follow customers home and watch them interact with the software back in the early days of Intuit. They said that this focus on how the customer interacts with the software is the reason why Intuit has been so successful. This serves as an industry example of contextual inquiry in practice where the designers actually studied the user. I also thought that some of the further refinement in the readings on how to conduct interviews for contextual inquiry was very perceptive. Most striking was the identifying work structure from the summary. Many things in our daily life are often summarized into one or two sentences but encompass so much more.

David Jacobs - Feb 07, 2008 03:05:47 am

One of the things that struck me as I read Beyer and Holtzblatt's article is that they go to great lengths discussing how important it is to have a simple mindset going into an interview, and then go straight to describing a complicated system of interview technique. They assert that if you are too busy remembering a big set of rules about how to act, you'll be too distracted to really pay attention to the user sitting in front of you. This leads me to ask, what exactly are they describing if not a large set of rules about how interviews should be undertaken? I'm not suggesting that the set they present is unsatisfactory, in fact, I think their recommendations are actually quite good. I guess it's more a matter of familiarity and balance, rather than whether or not the interview procedure is well defined. It actually kind of reminds me of the anthropologist technique for studying cultures, which is to immerse yourself in an attempt to understand the user by becoming a user.

Gordon Mei - Feb 07, 2008 02:19:42 am

The Beyer & Holtzblatt piece covers the "interviewing" of customers and users to figure out how users work, largely by following their routines and workflow, much as an apprentice learns from a master through observing the vocally annotated process over abstract lecturing. In performing a given task, the user can remember what aspects worked well and particularly what worked poorly. This is crucial in a designer's method of properly understanding unique user gripes about a product. This is why, in software, there exists beta testing and other forms of pre-release versions of products where a user actually works through and uses the product, and thereby "interviews" when coupled with feedback and bug reports. It is during these trials that it becomes far clearer what works and what doesn't, beyond the initial assumptions by the designer.

Edward Chen - Feb 07, 2008 03:22:44 am

To me, what Beyer and Holtzblatt wanted interviewers to do for contextual inquiry, especially for the first time, seemed a bit unreasonable. Sometimes it's just difficult to recognize a lot of the cues and pitfalls that may lead the interview astray. In addition, a lot of the additional informations was in the manner of how the customer reacted in terms of how their expression was like or how the wording of their sentence was like. A rookie interviewer doing contextual inquiry would have a hard time focusing on the many aspects of contextual inquiry while maintaining the awareness to pick up things like that. Overall, what Beyer and Holtzblatt said made sense and was probably very effective. However, it seems like the best way to get better at contextual inquiry would be through experience and being able to naturally do what Beyer and Holtzblatt mentioned over the course of many experiences. The only part that really stood out to me as being awkward and weird would be the contextual inquiry proper section in which Beyer and Holtzblatt suggested the degree of nosiness one should be during the interview. It just really surprised me about the degree of detail that they suggested the interviewer to have.

Hsiu-Fan Wang - Feb 07, 2008 03:23:38 am

I think it is naive to pretend that a contextual inquiry (which they refer to as an "apprenticeship compressed in time") can truly capture user problems. I suppose it is particularly effective at simply discovering egregious errors, but most nontrivial human tasks seem to require an element of instinct. Many of the examples that are mentioned seem to involve humans acting as some sort of automata, mindlessly filling out forms, but the process seems to break down for anything involving workers given very general tasks or goals. I imagine someone watching me use Photoshop over my shoulder as I perform a task would ask many questions about the selection tools or how a tool is used, but discovering why a set of tools are used under one condition, and another under different sets would probably require an inordinate amount of time (selecting a region is merely a small step in editing but can be executed in different ways) and requires multiple runs to discover the different approaches that one could take.

William tseng - Feb 07, 2008 09:56:01 am

I think what the article has described in terms of the apprenticeship model and partnership are what would be "ideal" forms of interaction between interviewer (developer) and customer. When I say ideal I mean the article assumes that a given company has unlimited time and resources to spend that level of detail to sit down with one customer at a time and pay attention to all of the details of their work structure. I think while the article is correct in describing the benefits of using this system of interaction in a realistic project again I don't believe this should always be the way we approach an interview. The described methods seem more applicable for when you are looking to design something from scratch. The whole concept of looking at work structures and thinking about ways to imporve them implicitly implies you are creating something new to help existing problems. There will be times in the iterations of a product where it might be more useful to get a LOT of feedback instead of several detailed feedbacks on a product. Those would require less personal approaches such as a survey or strict interview list of questions.

Kai Man Jim - Feb 07, 2008 10:31:37 am

This article has some good points like people are using abstract all the time, saying "usually" is the key work for using abstract by applying something they already known to another thing that they assume to work the same way. It is good that the interviewers keep asking questions for proving what people abstractly thinking does work, and it also help the people to pull out their memories. But then the article keeps saying what relationship is bad to use for interviewer to ask questions from customers. Some of them will not getting an expected answer since the relationship was wrong at the beginning, the other was just making the customers scare in the way the questions are asked. So I am confuse now, since I know how people think, and I know what is not good to do when asking commons from customers, then what should we do to get good result from customers?

Timothy Edgar - Feb 07, 2008 10:58:21 am

The bit about adults being unconsciously trained to summarized is quite true. Even just catching up with friends, we often times have little to say even though many things happened since we last saw a friend (especially friends you see infrequently). The cues for people thinking about the abstract is interesting about where people put their eyes, which can be useful to catch people from not including a lot of important detail. The models were an interesting concept since it conveys a lot of information similar to the article about natural cues. I was a bit surprised at how they had so much theory at the start yet just ended with a detailed how-to list. It caught me off guard a bit since I figured it would talk about high level principles to follow and accomplish a lot more and perhaps explore other scenarios. The avoidance of bad models such as expert/novice and guest/host is a good piece of advice to look at how you are interacting that may put off people. I imagine many of these concepts can relate to different aspects of the customer feedback cycle.

Bruno Mehech - Feb 07, 2008 09:31:41 am

I think that the pitfalls that that Beyer & Holtzblatt talk about is the most useful information to get out of this reading. Pitfalls that I found particularly interesting and which I hadn't really thought of before were the ones that had to do with focus. Whenever I am in a situation where I'm learning something new by observing someone I always go in with several assumptions and ussually find myself doing pretty much all the things that Beyer & Holtzblatt warn you not to do. Something else I found interesting is how they say that the interviewer should constantly be switching from a master apprentice to a interviewer-interviewee relationship and to be very strict about when switching roles. Overall though, I think the most useful information are the things they tell you to avoid.

Lita Cho - Feb 07, 2008 12:15:10 pm

The main idea I took from Beyer & Holtzblatt chapter is going in with a open mindset, and back tracking the interviewee to gather all the details. It seems like the author wants us to go in with a child-like mindset, thinking everything is new. I also agree that while interviewing someone, it is easy to fall your own assumptions. Listing the various pitfalls was very interesting. I believe that the Master/Apprentice Model works well when interviewing someone to gather information about their job or daily life. However, this method might be time consuming. To go around and interviewing various people would take a lot of time. Also, to get the full effect of this model, it seems like you would have to watch the customer for long periods of time.

Adam Singer - Feb 07, 2008 12:48:56 pm

I found this article quite useful; not only for this class, but for my every day life. Quite often I am interacting with customers trying to elicit their problems, requests, and confusions. As this reading pointed out, I often found myself taking on the expert/novice role, acting more as a tech support representative than an apprentice. I suppose this role comes naturally with the territory: we as designers know a lot more about a particular product than the lay person. However, we must notice ourselves falling into that role in order to salvage valuable interview time. I felt a lot of my time was wasted helping customers through a technical problem rather than discovering their wants and needs for the product I was designing.

I suppose the two main aspects of conducting a successful interview I most need to work on are 'focus' and 'interpretation'. Losing focus is easy for the aforementioned reasons, and interpreting the data I collect during an interview can also be quite challenging. In order to improve upon these two areas of weakness, I will take the advice given in this article to heart the next time I am eliciting data from a customer.

Andrew Wan - Feb 07, 2008 12:58:41 pm

While much of the reading seemed self-evident, it did a good job of putting the reader in the right mindset for interviewing users. Knowing how to ask customers effective leading questions and how to interpret their responses is crucial for getting the most out of interviews. In the end, it seems like building some ad-hoc working relationship with people is most helpful in getting truthful feedback. The faster you can associate with the customer, the better your ability to tell when they are tacitly saying "no", or when your questions are worsening the interview.

Brian Taylor - Feb 07, 2008 01:02:59 pm

I did not think that so much was involved in dealing with collecting research from observing someone at work. At the same time, it does make sense that many different relationships could arise between interviewer and interviewee, many of which are totally useless to the designer. I found the entire discussion of the master/apprentice relationship a rather accurate model of what would be the ideal way to learn from a customer. I found the reading interesting, though, in how it began by stating that to conduct a good interview (and obtaining valuable information), the interviewer only needs to do what feels natural to him or her. At the end, however, we have a not-extremely detailed but still fairly explicit list of some do's and don't's. The list of indicators of a bad-interview is probably one of the most useful sections of the reading, while the earlier portion works primarily to share a few different types of relationship models and encourages us, the designers, to choose the right one. Although we learned that observing a person doing their job may be a useful way of gathering data, I did not realize that a person could potentially be unable to describe their common everyday routines without physically doing the work. I found the portion about the secretary describing her morning routing particularly insightful about how people do much of their everyday activities fairly instinctively after the routines have developed. Often, only while the person is doing the work, is he or she able to actually discuss the task and the reasoning behind the actions he or she takes while doing it.

Andry Jong - Feb 07, 2008 01:13:57 pm

Beyer & Holtzblatt, in their article, basically, tells us that there are a lots of ways to get information from people during interview. It is very true that we can learn so much about customers by seeing how they work on things. Moreover, when we - interviewer - see people working, we can generate more effective data rather than the summary that customers give us. Another good thing about being in the work place as the customer is working, we can get more detail in what they are doing, since we can ask questions when we see some action that we do not know the purpose of doing.

I believe, this might be a good way of designing a good user interface design if we are only focusing on a certain type of people or a certain company. Since we can observe how they actually work, we might be able to build the more suitable design for them.

Yang Wang - Feb 07, 2008 01:22:39 pm

Beyer and Holtzblatt's paper discussed both the relationship models to interact with consumers and also some useful technique and procedure to perform the interaction. It didn't specifically state which one is a better model, in stead it gives the advantages and disadvantages for each of them. I think it meant for readers to pick the suitable model for their own purpose. Master / apprentice is a god example, while it is good for teaching and getting apprentice to know stuffs fast, it also limits how feedback could be coming from apprentice. About the procedure, it seems models all follow a general rule. That is ask question, listen to answers, watching the actions, comprehending ideas and give feedback back to consumer. These are pretty standard and effective. This paper supplied some good examples regarding how to do each step as well, like the questions and answers it gives.

Richard Lo - Feb 07, 2008 12:57:19 pm

It felt strange reading this article, almost as if it were unnecessary or a little too detailed about something that shouldn't be too strictly defined. Many of the previous comments already addressed this, so I won't go into it too much. But I agree with them that these things shouldn't be governed so strictly. People work in different ways, learn in different ways, interact in different ways. I feel most people would just have the common sense to fall into a method of observation that would work best in the situation they are in. But I guess it may require some experience and wisdom to be most efficient in going with your intuition.

That being said, I do agree that the master/apprentice relationship works the best when needing to learn quickly about a certain skill or trade. I don't agree with the reading that this relationship doesn't exist anymore, in fact I was in this type of relationship at my summer internship last summer when I was mentored by an experience staff engineer. In fact, pretty much all the time when you enter into a new position, you will be mentored and learn on the job much like a master/apprentice relationship. Most of life's skills are caught, not taught.

Johnny Tran - Feb 07, 2008 01:19:32 pm

I like how this article emphasizes the importance of communication between customers and designers. I wonder how much of this actually occurs in practice, especially for software design. My impression of general software design is that software is made first, and is then sent to QA. The article seems to advocate the reverse, an approach that would likely result in better software.

I found the communication guidelines to be very interesting; I never really considered the different relationship models and pitfalls of customer-designer interaction. If I am ever in a customer interview, I will have to keep these points in mind.

Jeremy Syn - Feb 07, 2008 01:38:30 pm

I agree that the master craftman and apprentice model is a very effective way on learning a skill. This model can also apply to acamedic studies; the more a student practices problem sets on his own and watches the professor solve them as well, he will eventually get better at it and learn the material. This model is also important in terms of satisfying customers because we must observe the customers at their work and see how they interact and also what difficulties they have that can potentially be solved.

EricChung - Feb 07, 2008 12:41:36 am

Certainly, if the focus is on the customer, the product will be better designed. I liked the ideas present, especially the part about how it's not just facts but the interpretation of the facts that matter. For me, although completely logical, it wasn't quite as obvious at first. The idea that the interviewer should act as an apprentice is a good one, I think and is surely would allow for one to get the most out of hte interview. I didn't quite see a solution to the problem of focus in the reading but I suppose I'll get clarified on this during lecture.

Maxwell Pretzlav - Feb 07, 2008 01:47:57 pm

I really like how the Beyer & Holtzblatt piece ties into the Task-Oriented design article. While the philosophy of task oriented design seemed all nice and good to me, the problem I saw with it was that it didn't specify any of properly understanding the task at hand. This article fills in the gaps and hows how by watching a series of similar tasks and using a specific technique to record and understand what a user/customer is doing, the designer can properly and fully understand a task in order to design for it.

Joe Cancilla - Feb 07, 2008 02:08:58 pm

I feel that Beyer & Holtzblatt are absolutely brilliant for using the Master/Apprentice model as an attitude to take when conducting a contextual inquiry of the customers daily routine. This would allow the customer to go about their daily task, while providing useful insight into their processes. This also sets up the customer as a partner in the inquiry as opposed to a mere object of study. Another great benefit I see is that it would prevent the designer from criticizing the way the customer does their work. Designers aren't hired to change the customer, but to change the product to meet their needs.

Jeffrey Wang - Feb 07, 2008 01:45:10 pm

The author outlines the methods of learning from the users: context, partnership, interpretation, and focus. The article provides us with some general methods to follow and what to keep in mind in an interview. I think one thing to mention is that user behaviors and habits often changes when the user is being observed. LIke what others said before me, there is a thin line between interviewer and the interviewee. I think it's important not to be too intrusive.

Also, while the author provides some straightforward guidelines for context and partnership, it is really up to the designer's judgement in terms of interpretation. The designer is recommended to validate it with the customers. However, if the designer is not careful, I see it is possible to inject ideas into the customers.

Diane Ko - Feb 07, 2008 02:20:07 pm

Beyer and Holtzblatt emphasize the importance of the relationship models when observing customers using a product designed by the observer. The aim in any relationship model is to go back to a more partnership based relationship in order to get the most productive responses from the user as to how they use the product and what they consider more important of the features. In the contextual interview structure, it seems like it would be somewhat difficult to get to the transition stage, shifting from the conventional interview to the contextual interview. Many times it's very easy to get stuck in a certain method of thinking and interacting, making it difficult to change the dynamics of the already established relationship between two people. If you don't shift properly to the transition stage at the right time, there may no longer be that possibility to do so later on once the roles of the two people have been set for a long time.

This article focuses on how important it is for the designer to really understands how users use their product. I found the focus triggers (surprises and contradictions, nods, what you don't know) to be quite interesting in that it's very easy to dismiss those triggers as nothing important when they actually indicate strongly what could be wrong with the design of the program. What seems intuitive to the creator/designer of a product can be very counterintuitive or confusing to the user of a program they don't have extensive experience with. Also, it's important to focus on what users do all the time to determine which features of the program should be exploited more. Some of the programs I've come across, for small businesses especially, seem like they were made without attempting to gather enough information on how customers will actually use the product.

Tam La - Feb 07, 2008 12:55:51 pm

What I got out of the reading is that Contextual inquiry is basically a structed field interviewing method, based on a few core principles that differentiate this method from plain, journalistic interviewing. Contextual inquiry is more a discovery process than an evaluative process; more like learning than testing. Contextual inquiry is one of the best methods to use when you really need to understand the users' work context. Many times, the environment in which people work really influences how people use a product. I thought the reading was interesting.

Jun Kang Chin - Feb 07, 2008 02:20:03 pm

The reading drives the point across that contextual inquiry should take of the form a master-apprentice relationship model. The idea that such an relationship model is very interesting and it's too bad it the article does not touch on other components of other relationship models that could work for an interview process. Also it's interesting of the concept that during the interview, customers has to voice their every action aloud. Now that they mentioned it, it's a powerful tool to see the efficacy of one's actions and may even lead to more streamline business processes for the customer.

Raymond Planthold - Feb 07, 2008 02:46:13 pm

The part I liked most about the approach B&H describe is asking a lot of targeted questions, which can lead to the user adding their own information, rather than a few open-ended questions that stump users who aren't prepared for them.

I did find it odd, however, that in the example given in the interpretation section regarding the user who kept a list of account numbers next to her screen, they did not mention asking her why she did so before coming up with a list of interpretations. It seemed rather out of place with the rest of the reading to only ask for input after analyzing the facts, rather than both before and after. The later examples worked better because they were simpler.

Megan Marquardt - Feb 07, 2008 01:38:29 am

The discussion of different relationships while gathering information from a client was very interesting. I never really thought of how different types of relationships infer different aspects, roles, and experiences about each person. The master/apprentice was especially applicable in that both people need to be engaged in what is occuring during a particle workflow or task. It was interesting that they did point out that the apprentice does not ask enough questions, because it is important to get as much information out of the client as possible. The interviewer needs to be able to ask which parts are important, why a certain task is performed, why a certain workflow is perferable over another. The other relationships to be avoided were also interesting to read, especially the expert/novice, since I would think that model would be avoided just because of human nature. It would be tempting to act too knowingly as the interviewer since they have much more knowledge about design, and the interviewer might try to assume things about the client's workflow. I guess if I were the interviewer, my trouble would not be in having to remember that I'm not the novice, but in having to become the apprentice, a seemingly harder task personally. Overall, I agree that in order to learn about the client's work, the relationship should be master/apprentice where the apprentice is slightly more removed, focused more on the design aspects as opposed to actually doing the work, and also where the apprentice isn't quietly watching, but asking questions and trying to learn through inquiry.

Zhihui Zhang - Feb 07, 2008 02:40:47 pm

The master apprentice model seems like a very good model for the interview process because it allows the designer to gain insight into how the users perceive the product. Having the user guide you through task that they normally performs is a good way to gather feedback. In particular, sometimes users will face issues with a product that they're not really aware of (or using hackish was to do various tasks). For example, I knew several people in the past that did not know how to enable double spacing in microsoft word and instead did all the spacing manually. To them, this seemed like a very natural way to do double spacing, and unless the designer saw this, the issue would probably never have surfaced.

Robert Glickman - Feb 07, 2008 03:00:59 pm

I like how this analogy works in analyzes a user's interaction with an interface -- the master-apprentice model works well, and it is interesting to see the expert in the technology become an apprentice. In particular, the examples were enlightening and the pitfalls illustrated later were particularly useful. I thought that the way the woman showed how she looked up the icon was interesting because it showed a method of documentation using easy-to-find icons which correspond to what the user is used to seeing. Overall, this article opened my eyes to observation of a user's experience and expressed many of the traps that I would have fallen into, as an evaluator of someone else's routine.

Mike Ross - Feb 07, 2008 03:14:24 pm

I was really interested by the subject of managing yourself and your relationship with the interviewee. It's an interesting read in that the general idea applies to every aspect of life, not just contextual analysis. If actually gave me a lot of insight into how to effectively convey information about methods and practices as well. Some friends and I teach courses on animation and using the ridiculously complex software package Autodesk Maya, and we're always looking for methods of teaching which are more exciting than mentioning every available feature, but more efficient and effective than simply letting them watch us build things on the projector. I guess the question it leaves me with is how you can set up a relationship where the students aren't just waiting for us to throw information at them, but are engaged and asking us how and why we do certain things. I suppose I'm wondering how to get them to be the analysts. This veers way away from task analysis and contextual inquiry, but it sparked my interest.

Siyu Song - Feb 07, 2008 03:15:21 pm

Beyer and Holtzblatt were very explicit in how to interact with a customer. I thought it was interesting that their article was effective because of a lot of the things they said to do during interviews with a client. For instance having the client giving instances of tasks or routines when they begin to generalize. Like the example of the secretary who says she begins her mornings by checking her messages, who actually ends up being able to add a lot more details by actually doing it. The authors conveyed their point much clearer to me because of this anecdote that served as a concrete example. The other good point I noticed was about how to maintain control of the interaction while letting the client lead in terms of giving out information about what features are most important.

Jeff Bowman - Feb 07, 2008 03:17:54 pm

The most impressive thing about the reading that to know your users, you need to ask them, but that the asking will not always provide you with good information. It seemed silly to spend too much energy worrying about how to ask users, but the examples he provides demonstrate the truism of design: Users know their routine, but may not be able to describe it.

After completing the reading, I also thought about how this technique could be repurposed for just about anything; managers want to know about their employees' workflow, building architects and civil engineers want to know about the people for whom they design, and the list goes on. And though I do admit that some of the points seemed arbitrary, the final paragraph drives the one-line message home.

<meta>As a side note, I notice that the format of the book is interestingly designed, and I had an easier time following it than standard long columns of text.</meta>

Zhou Li - Feb 07, 2008 03:28:24 pm

Beyer and Holtzblatt presented the master and apprentice model for contextual inquiry. They argued that while this model can provide an effective relationship for information gathering process between customer and design team, some modifications are needed for it to be useful in gathering design information. Out of the four principles, I think the “partnership” restriction is the most important one, because the designer has to interact with the customer to get design related questions answered. So the customer cannot simply act as the master; just teaching all their current practices to the designer. Since the purpose of the interview is probably finding a better solution or improvement to existing process. The designer has to lead the customer to elaborate on details regarding the design problem. However, the designer cannot control the interview process either. The best way to get detailed information plus past related experience from target customer is to put them in their familiar working environment. That way they have all the visual and mental cues to generate precise description of the problem.

Paul Mans - Feb 07, 2008 02:30:27 pm

I think there were many guidelines articulated in this article that would be useful when performing Target User analysis. Getting the customer/user in their own element with the master/apprentice relationship makes a lot of sense to me. Bringing the customer/user back to the concrete, increasing the number of details generated in the interview, and keeping the customer from attempting to summarize are some of the benefits of the master/apprentice interview that resonated with me.

Another thing I liked about this article was the author's insistence that the interviewee introduce design ideas to the customer during the interview itself. This follows a theme that has appeared in numerous readings that getting feedback early is almost always a good thing. For one, getting immediate negative feedback on an idea will reduce the time you spend on it. Secondly, I think in general it is just a good practice to make your ideas open to criticism early because it will reduce your attachment to them. It is all too easy to view criticism of your ideas as a personal slight and this type of emotional reaction is a major destructor to a team.

In a lot of ways the ideas presented in this reading and others throughout the term have seemed like common sense. However despite how obvious some of these techniques sound I am completely convinced that I don't exercise them nearly enough because it is easy and natural to fall into other patterns of interaction. Many of the readings so far have described in detail a process that is by nature amorphous and fuzzy. I think the greatest challenge in this class is will be to try and adopt some of these "scientific" approaches to design while not imposing so much structure that creativity is stifled.

Pavel Borokhov - Feb 07, 2008 03:24:58 pm

The reading provided a very thorough overview of the way inquiry should be handled by professional designers who are looking to improve the interface for a group of users. It also underscores how painstaking such a process is, and how "getting it right" is quite complicated and requires a significant commitment of time. Personally, I would be interested in seeing how "real" a user's actions are when there is someone observing them. The master/apprenticeship relationship is an interesting model and certainly does a good job of representing the relationship that should exist between the designer and user, but I really wonder how much the user, especially due to their perception of the designer as the "expert", would alter their behavior as a result of being observed. I also found it interesting that the author stated that it was important to have the user provide feedback on the designer's interpretations of the behavior and work that they have observed. Lastly, the triggers mentioned in the focus section seem like traps that would be extremely easy to fall into and every designer should do their best to avoid.

Max Preston - Feb 19, 2008 02:49:46 am

I thought it was interesting to read about the technical aspect of designing a user interface as opposed to the conceptual ideas of how people learn to use a given interface. On the other hand, I thought that a lot of the details were too specific to really be very useful unless you are working on the same system as the one in the article. Also, it seems like a lot of the ideas presented in the article might not be so relevant today unless you're working on really low level stuff and I'm not totally sure if the structures described in the article are exactly the same as the ones used today. In any case, development tools like Android handle most of the brute work so you don't have to go to the trouble of figuring out how to program a simple button.



[add comment]
Personal tools