Contextual Inquiry-Group:Oh Mah Gah!
From CS160: User Interface Design Sp12
Project Idea Change
Reason for Change
After interviewing a few people for our drawing app, we decided that drawing through the Kinect medium would be a inaccurate (there is a lot of jitter, and even if normalized, still wouldn't allow the level of accuracy a mouse or tablet would allow), tiring on the arms (many artists/designers go over the same lines multiple times until they get it right), and tedious (learning the various ways to gesture commands the artist/designer was already used to with keyboard shortcuts and tablet buttons). Although the color selection idea received positive feedback, no one felt that this was enough to overcome the negative aspects of drawing through a Kinect. We considered doing a whiteboard idea, but noticed at least two other groups were implementing this idea already.
Idea Selection
We decided to implement a kitchen recipe helper, parsing steps from a recipe and presenting them in an easy to interact interface. We feel that the kitchen environment lends itself very well to a Kinect application, where no physical contact needs to be made with the application in order to control it. This solves the problem of having to wash your hands to keep your laptop/tablet/book clean. By being able to track multiple recipe steps at once with the help of global timers, cooking efficiency can be increased. Using a combination of voice commands and gestures, the cook is ensured that he never needs to physically make contact with anything in order to interact with the application. This application targets anyone who follows recipes in the kitchen, from an experienced cook to one who is just starting to learn.
Contextual Inquiry
Group Members
Mark Peng - Target users, problem and solution overview, interface design, interview, final wiki page
Bei He - Analysis of tasks, interface design, sketches, final wiki page
Yian Shang - Interview, competetive analysis, interface design
Eric Mao - Interview, task analysis questions, interface design, final wiki
Target Users
Anyone who wishes to follow a recipe in the kitchen can benefit from our application. Due to the one recipe at a time limitation, though, amateurs looking to learn to cook would be drawn more to our application than more advanced cooks who prepare multiple dishes at at time.
| Gender | Cooking Skill Level | Tech Savvy-ness | How often recipes are used | How often they cook | Additional info | Reason for choice |
|---|---|---|---|---|---|---|
| Male | Intermediate | Advanced | Rarely | Daily | This user rarely relies on recipes, likes to customize recipes when trying them out, and prefers using a laptop if following a recipe (for convenience reasons) | As a tech savvy person who cooks often, he was chosen as someone who could be a typical user of our application |
| Female | Intermediate | Intermediate | Not often | Usually daily | This user generally doesn't use recipes, and whenever she does use one, calls her mom (doesn't find them online). | This user was chosen as our average kitchen cook. |
| Male | Intermediate | Intermediate | Not often | Often | This user occasionally relies on recipes when feeling creative. He is meticulous in following steps and make sure ingredients are all the right amount. | This user was chosen for his meticulous attention to detail when following recipes. |
Problem and Solution Overview
When following recipes in the kitchen, the current tools available are both inconvenient (requiring clean hands if you do not want to get your devices dirty) and leave much to the user (requires the user to set timers). Our Kinect application will use a combination of gestures and speech recognition to provide an interactive, hands-free recipe viewer. The application will also make creating and tracking multiple timers while following the recipe a streamlined process, starting timers and notifications as the recipe needs them.
Interview Descriptions
Interview 1
Process used: I asked the subject to choose a recipe and follow it, observing and asking questions when needed.
Gender: Male
Cooking skill level: intermediate
Tech savvy-ness: pro
How often they use recipe: rarely
How often they cook: daily
Tasks performed: grilling, mixing sauce, marinating meat, cooking meat, combining
Environment: Apartment kitchen
Pick a recipe, explain why they did each task:
He started out with placing laptop on the counter furthest from stove (why? want it as far away from all food prep as possible so that laptop doesnt get dirty)
Pulled up recipe, took out all ingredients and measured them ahead of time (why? easy access when they need ingredients, and make sure they have everything)
Then, he read recipe for things that would be combined anyways without prep (save number of bowls needed to hold ingredients)
Started with step 1, combining sauce to marinate meat in
Put meat in the sauce to marinate, looked at clock to see approximate time (why? not a very time sensitive task, not worth the effort of setting a timer)
Chopped eggplant/garlic/vegetables (next step)
Cooked all fresh produce, then set aside when almost done
After placing vegetables aside, start cooking meat with the sauce
Taste test the sauce and adjust (should leave room to make notes on recipes to customize for yourself)
Once meat done, add in vegetables and combine
Upon reaching the last step, he closed his laptop.
Interview 2
Process Used: I asked the user a series of questions to gather personal details about them
Gender: Female
Cooking Skill: intermediate
How often they use recipes: Not very often, and whenever she needs one, she calls her mom
How often they cook: She cooks often for meals, usually once per day
Tasks performed:
She uses the stove mostly and the oven occasionally for baking. She has a rice cooker, a pressure cooker and various other devices in the kitchen. The pressure cooker is extremely loud when its contents are ready and can scare potential roommates. Generally for dinner she’ll wash vegetables, chop them up and add spices. It would be useful for her to know which spices she needs to use (have pictures of them) and how much. She says that sometimes she makes mistakes in the kitchen and she’d like to know what can be done to fix them.
She also occasionally cooks for large groups of people and could use an easy quantity multiplier that would tell her how much she needs of each ingredient when she’s cooking for X people. She wants to be able to adjust her cooking to suit their palettes as well and cater to them in case they have allergies.
Where she would put the Kinect: outside of the kitchen, on the table (her kitchen is small)
Environment: Small apartment kitchen
Pick a recipe and perform actions:
She decided to make scrambled eggs. First, she opened her refrigerator and obtained an egg. Then she found a bowl and cracked the egg into the bowl. Afterwards, she went to the sink and washed her hands because she didn’t want the raw egg to contaminate the rest of her kitchenware. The she used an egg beater to whip the egg. A pan was already sitting on the kitchen stoves, so she just poured some oil into the pan and brushed the oil around a bit so that the entire pan was covered. Then she poured the raw egg into the pan, waited for it to simmer and she flipped the egg when it looked like one side was near done. It was obvious that she already knew the recipe quite well (she does this daily) and so most actions were familiar to her. However, the length of time required to cook each side of the egg is still a guesstimate for her each time she cooks and she says if she were cooking something more complicated it would be useful to have detailed instructions on when to flip the egg, when to stir etc.
Interview 3
Process Used: I asked my interviewee questions as he went about cooking. I observed how he performed the actions and asked him why he did certain things.
Gender: Male
Cooking skill level: intermediate
Tech savvy-ness: moderate
How often they use recipe: not often
How often they cook: often, 5 days a week
Tasks performed: cutting, sauteing, mincing, combining sauces
Environment: Apartment kitchen 20ft x 8 ft
Pick a recipe, explain why they did each task:
Grilled pork chops, onion, tomato stew: The user prepped for the meal by defrosting the pork chops two days before. A day before cooking, he marinated the pork chop by mixing soy sauce, soy paste, hoisin sauce in a bowl then placing the pork chop inside to let it marinate overnight. On the day of the cooking he prepped the cooking area by cleaning all cooking surfaces. He washed the tomatoes and onions then cut the onions into quarter pieces followed by slicing the tomato into eighths. He first allows the oil to heat up and then sautees the onions. The user is very detailed about each step being very sequential. He then adds the tomatoes into the sauteed onions and cooks until tomatoes are nearly done. Next he gets another pan and grills the marinated pork chop. He flips the pork chop often and is very attentive to make sure the sides are not burned. He cares about timing a lot. Finally he combines all the items together in the sauteed onion pan and cooks for 10 minutes, checking the time at the 7 minutes, 9 minutes, then 10 minutes mark. The dish is finished.
Notes
Interviewee 1 did not use any "laptop friendly" versions of the site, and had to lean in close to the laptop due to the small text. He also mentions having to read all of it ahead of time because he needed to make sure he had the correct kitchen appliances and cooking-ware to make the dish in addition to having the ingredients.
Interviewees 1 and 2 both wished for more customization, either by changing the recipe itself or by changing the number of servings to make at once.
The cooks all followed the instructions in sequential order in the recipe. If a step took too long, the laptop screensaver would come up and that would cause some hassle for the cook. Users always had to have clean hands before interacting with the medium containing their recipe.
Task Analysis Questions
1. Who is going to use system?
Beginner cooks who love exploring new recipes, students who are tired of eating ramen every meal, couples wanting to make cooking a more interactive and fun experience. People who want more organization in the daunting task of cooking for the first time.
2. What tasks do they now perform?
When people want to follow a recipe online, they bring their laptop into the kitchen and have to scroll either with a touchpad or mouse to see the next recipe. They have to click around to select timers if they want to keep track of time. Often times their hands are occupied or dirty because they are working with food.
3. What tasks are desired?
Hands free scrolling through recipes, step by step instructions on a big screen since the user may be far away from the laptop sometime. Hands free timers that keep track of time left for each task (ie. bake for 15 min, simmer for 2 min.).
4. How are the tasks learned?
Large buttons with minimalist design will bring focus to the main tasks available at each step. People will be guided with clear helper tips when the user utilizes this application for the first couple times.
5. Where are the tasks performed?
The tasks are performed in the kitchen since this is a cooking application near stove tops, the refrigerator, sink, and table so safety considerations have to be made. Large text will allow users to place laptop further away from work area to prevent spillage onto computer, increase safety, and decrease clutter in an already busy environment. The work space will probably be in an area of around 15 feet long by 10 feet wide. Display and Kinect will be placed at the end of this work area which is still in the range of the Kinect sensor.
6. What’s the relationship between user & data?
Data will be accessed via a browser to grab online recipe data. Therefore, users will need an internet connection initially but can download the recipe for offline browsing. We do not foresee any major privacy concerns because all data we use is publicly available and accessible. The only slight concern we may have is the function that allows users to download and save frequently used recipes. Since our data, recipes, are not sensitive information, no authentication or password protection will be implemented to access recipes. This would cause unnecessary actions that complicate user flow. In our demo, recipes will be manually inputted to demonstrate the functionality of Kinect and allow a greater focus on user interface rather than parsing web pages.
7. What other tools does the user have?
The user needs a display large enough to see the recipe instructions. Since the kitchen is a busy and dangerous area, it is important to have a safe and secure location to place the Kinect and display.
8. How do users communicate with each other?
This is not very applicable to our application since other communicating with each other is irrelevant to the Kinect application.
9. How often are the tasks performed?
The Kinect is accessed often to swipe to the next instruction in the recipe. Since this will be performed often, we seek to make this as unobtrusive and natural a motion as possible.
10. What are the time constraints on the tasks?
Most recipes are completed within 30 minutes. The application includes automatic timers to alert the user when certain tasks are finished.
11. What happens when things go wrong
If an alarm does not go off to alert the user that an item is done baking or cooking, people can have their food burned or in the worst case a fire could occur. People may not be able to hear the alarm if they leave the kitchen or the ambient noise too high. We will make sure to provide a loud enough alarm and have redudancy systems to make sure the user hears the alarm even if the person is not there at the moment when the first alarm rings. In the end, the user can’t depend completely on this application just as if the user did not have this application’s timer, they still need to have to be alert and aware of their surroundings (ie. whether things are burning). This application will alleviate the things that people need to keep track of.
Analysis of Tasks
Easy:
- Viewing ingredients while cooking requires the user to return to the recipe book and then locating the ingredients section. Our application allows the user to simply scroll up the steps list to view the ingredients without having to leave his current task unattended.
- Going to the next step requires the user to physically change pages and then relocating the step's location upon returning to the book. But in our application the user can simply gesture to display the next or previous steps.
Medium:
- When cooking with traditional recipe books, the user must browse through the table of contents or index and then navigate to the correct page where the recipe is. Our application allows the user to simply say a voice command that directs the selection to that particular recipe.
- Setting a timer in real life necessitates locating a device that supports time tracking, inputting the correct time into that device, and then checking its progress as time goes on. However, our application automatically creates a virtual timer that displays the progress of the timer and then notifies the user upon completion without needing the user's attention or action.
Hard:
- In real life, the user must manually photograph the dish if he wants to document the success of his cooking endeavors or to share his glory with his companions. But with our application, this process is smoothly integrated into the final procedure by allowing the user to take a photograph with the advanced Kinect RGB camera and automatically share this joyous moment with others.
- Without the aid of our application, tracking multiple timers can be a difficult, if not impossible, task for users using complicated recipes. Our application is able to display numerous timers that are easily identifiable and easy to modify or remove.
Interface Design
The application starts up with a recipe browser (with preloaded recipes; possible to sync with online databases in the future). On the left is a list of recipes that the user can navigate using gestures. Once a recipe is hovered over, its description and list of ingredients will be listed on the right side (Figure 1).
Possible Gestures:
(static) Right hand held at an upward angle: Move up in the recipe list at a rate relative to the angle.
(static) Right hand held at a downward angle: Move down in the recipe list at a rate relative to the angle.
(dynamic) Right hand swipe to the right: Choose currently selected recipe.
(dynamic) Right hand push: Same as above.
(voice) Saying the name of the recipe focuses on it (but does not select it).
Note: what gestures to use are not finalized as extensive testing is required to narrow down the best possible choices.
After selecting a recipe, the user will be taken to the recipes step by step walkthrough view. Here, the user is given the steps to the recipe one at a time and will switch at the user's gesture when that task is completed. The list on the left displays the steps of the recipe and highlights the one the user is currently on. The right side is a large display of the current step in text format. The reason for the large viewing area is that most users will be using the application on a television or laptop that has a small screen size. The user can signal to the application to go on to the next step using gestures (Figure 2).
Possible Gestures:
Because the user will be cooking as the application runs in the background, gestures will have to be distinguishable from the gestures made when the user is cooking. For this, we plan on having the user raise the left hand over the head to indicate that a gesture is about to be made and then make said gesture with the right hand.
left-hand over head + right hand right swipe = go to next step.
left-hand over head + right hand left swipe = go to previous step.
left-hand over head + right hand angles (same as the first view) = navigate through the steps without changing the current step (to look back or ahead in advance).
Some steps of the recipe will require that a task be done for a certain amount of time: for this we have a timer feature. This feature activates when the application finds a time string in the current step. It will then automatically add a timer to the bottom of the right panel when the user chooses to move to the next step (Figure 3). However, sometimes the user will not need the timer or want to pause it. Because selecting individual timers with a cursor is very difficult on a small screen, we decided to use the speech recognition feature of the kinect to allow the users to use commands such as "pause timer 1", or "remove timer 4". Once the timer runs out, the application will alert the user with sound and text as to which step has just finished.
After the dish is complete, users will be taken to a new screen that allows them to take pictures of their dishes with the Kinect to share on Facebook or keep as memories.
Here is the graph of the states and actions for this applications. Note that these are actions and not gestures because the actions are fixed while the gestures can be changed if they don't fit the action they describe.
Tasks
Task 1: Choosing a recipe
method 1: Gestures
If the user is unsure of what to make, using gestures to browse the recipe list while looking at the ingredients and information on the right is the best option. The user must then gesture to select the recipe.
method 2: Voice
If the user knows the name of the recipe, then saying the name will bring the current focus to that recipe. The user must then gesture to select the recipe.
The following figure illustrates this:
Task 2: Going to previous/future steps without changing the current step
The user can choose to view past (or the ingredients, which can be thought of as step 0) or future steps. To do this, the user needs to scroll up or down with the right hand while the left hand is over the head (the indicator for the application to accept gestures). While on a different step, the right side of the screen will change to the content of that step. However, the user cannot move to the next step using the swipe right gesture when not on the correct step. To go on with the cooking, the user must return to the correct step (which is highlighted) and then use the corresponding gestures to advance.
The following figure illustrates this:
Task 3: Correcting a timer
If by some mistake, the parser for the application makes an error and adds an incorrect timer, the user can modify it by doing one of the following:
method 1:
Remove the timer completely if the timer is not needed. This is done by saying "remove timer 1 (or the number/id of the timer)". If the user wants to add another timer, saying "add timer 30 minutes" will add a new timer that starts at 30 minutes.
method 2:
If the timer is simply incorrect, then the user can correct this by saying "set timer 1 30 minutes" to reset the timer to 30 minutes, which is presumably the correct time.
The following figure illustrates method 2:
Competitive Analysis
No similar Kinect applications were available; only video games on cooking exist. However, we found several websites that guide people along as they cook:
Epicurious’s mobile app (http://www.epicurious.com/services/mobile)
- Target User Group: According to the byline on Epicurious’s site, the service is “for people who love to eat” and there seems to be a focus on those who prefer healthier food. The app in general seems to be targeted towards those who are interested in cooking and want an easy way to browse recipes. Their target user group is broader than ours -- they cover a large gamut of cooks, both from those inexperienced to those who have been cooking for a while.
- Functionality: The mobile Epicurious app lets users choose amongst a large collection of recipes and can generate a shopping list based on ingredients the user needs. For each recipe, the user can view the instructions, photos and reviews.
- Usability: The Epicurious mobile app is very convenient to take recipes on-the-go. It just offers a list of recipes for the user to choose from and follow. Our idea will improve on it by allowing users to set up timers while they are cooking, which will be easier with the Kinect if their hands are covered with food.
Allrecipe’s kitchen-friendly mode (http://allrecipes.com/)
- Target User Group: They cover a huge amount of users, ranging from those with no experience to those with a lot of cooking experience. Their target user group is bigger than ours -- we mostly cater to inexperienced cooks, whereas they cater to everyone.
- Functionality: The recipe (both ingredients and directions) cover the entire screen real estate, allowing for those standing in a kitchen to view it easily. Our idea will also try to use as much screen real estate as possible so that the cook can see better, but we will also provide timer functionality for those who are not as experienced.
- Usability: The kitchen-friendly view of the Allrecipes site allows users to view recipes easily on a laptop in the kitchen.
myRecipes (http://www.myrecipes.com/how-to/video/making-an-omelet-10000001614210/)
- Target User Group: Again, their target user base is broader than ours and they want to cater to everyone.
- Functionality: There are videos instead of text recipes, but video transcripts are provided. There is no clear-cut ingredients list. However, this makes it difficult if the user doesn’t want to sit through a long video, so we will provide an ingredients list and instructions to give some more organization to the user.
- Usability: Watching a video can make reproducing the activity easier, since we can see physically what is being done to the food.
Simply Recipes (http://simplyrecipes.com/tag/Bacon)
- Target User Group: Their target user group seems to be anyone who enjoys cooking rather than our smaller target group of the inexperienced.
- Functionality: The site provides an index of recipes and clicking through to each recipe provides a picture, a list of ingredients and a set of instructions. The instructions are simple and not overly-detailed.
- Usability: Since the instructions don’t always fit on the screen (sometimes they require scrolling down to see everything), it is harder to use in the kitchen if the user’s hands are full and they don’t want to touch their laptop.
Food.com (http://www.food.com/recipes)
- Target User Group: Again, a broad target user group.
- Functionality: The site sorts the recipes by appetizers, beverages, breakfasts etc. Once you click through to a recipe, there is an ingredients list, directions and nutritional facts. They do not all fit on to one screen without scrolling. Thus, navigating the site while in a kitchen may be hard when users are reluctant to touch their laptops.
- Usability: While ingredients and directions are run-of-the-mill, the nutritional facts were not seen on other sites. Nutritional facts can be very useful for the user actively trying to eat healthier foods. We will provide better navigation in our app, as each instruction will take up most of the screen real estate and using the Kinect to navigate will not require touching any pieces of technology.
Summary: Most applications cater to a broad group of users, ranging from inexperienced to experienced cooks. However, our target user base consists only of those who are not experienced at cooking. Some of the competition attempt to provide screens that are friendly for kitchen-viewing (mobile apps and kitchen-friendly modes), but many do not address the difficulty of navigating recipes while working in the kitchen. Also, none of them provide timers to aid the novice cook in keeping track of the length of cooking time.
