PilotStudy-Group:J. S. & Associates
From CS160 User Interfaces Fa06
Contents |
Introduction
We are implementing a system that automates the sushi ordering process. The Anoto pen is used to wirelessly and instantly send food orders from waiter/waitress to the chefs in the kitchen. The system additionally computes statistical data and processes customer payments. We therefore have three sets of target user groups: waiters/waitresses, sushi chefs, and cashiers. This system aims to improve the efficiency of sushi restaurants and thereby increase their profits. It improves efficiency by: (1) simplifying the food order forms used by waiters and waitresses; (2) shortening the duration from when orders have been placed to the time food is delivered; (3) recording statistical data which can be used to monitor and improve sales in the future; and (4) providing functions cashiers can use to process customer payments.
The purpose and rationale behind the experiment at this stage is to obtain feedback from the different user groups regarding the usability of the interactive prototype we designed during the last assignment. Areas of concern we hope to improve are: (1) visibility of the provided interface (we added a color scheme to the Sushiman and Cashier menus to help distinguish orders); (2) ease of navigation through the Sushiman and Cashier menus; (3) adaptability to what systems are already in place (here we have merged a traditional food menu and order sheet to create a new order sheet for the waiter/waitress that resembles interfaces that are already familiar).
Implementation and Improvements
-Features added and improved-
For Sushiman's GUI:
Last time for the interactive prototype assignment, we could only finish the part to display the items that came in from the various orders. We worked on the displaying multiple common items on the left side of the GUI, and we almost completed it for last assignment but it wasn't perfect. This time the displaying multiple items works perfectly. Moreover, on the right side of the GUI, there were orders sorted out according to order numbers. However, we couldn't finish combining common items together per order (ie. ebi, hamachi, ebi --> 1 hamachi, 2 ebi) for last assignment. We have completed it as well. Additionally, we also added the functionality of keeping track of the time: the elapsed time from which the order came in to till now, so that the sushiman can set some priorities. The biggest change in the sushiman's GUI was the emergence of the "update" function. This functionality serves to resolve the problem that the TAs worried about: what if some other orders come in while the sushiman makes items, adding item to the common item table and confusing the sushiman. The "update" function updates the next set of orders only after all the orders on the screen are off the screen. There is also a feature in the GUI for the sushiman to input to set the maximum number of orders at a time. There is some concern with this however that this update function might be inefficient due to the fact that the next set of orders are delayed unfairly if the current set of orders are delayed for an amount of time. We are still discussing on this. Some people suggested to add two sets of orders at one time on the screen with different colors to distinguish them.
For Cashier's GUI:
There were a lot of improvements to the Cashier's GUI. We added the functionality of keeping track of the elapsed time from when the the order is served to the customers to current time. Also, for last assignment, we finished up to the part where the "details" button only displays the order number, items and their corresponding prices. However, this time we added the field for subtotal and total for each order, as well as for all orders per table. Moreover, there only were present the "pay" button and "delete" button. However, since we need to keep track of the elapsed time from when the order is served, we added a "served" button to make this functionality. This is to be implemented side, but we are thinking of actually printing out the check (receipt) through the printer.
For Paper (Waitress) UI:
Last time, we couldn't quite work out the linkage between the pen and the Sushiman's GUI and Cashier's GUI. However, we finally solved the issue. Since we haven't gotten the handwriting recognition to recognize numbers for number of items ordered and also for writing in the table number. For now, we have implemented the functionality by making the user (waitress) to tap on the item box or table number box several times to increment the number of items ordered or the table number. For the handrecognition part, since we are only dealing with numbers, not letter, getting the tool won't be that difficult (matching only 10 digits). To implement this, it would require us to split the field boxes into digit boxes to write each digit in. There are also two fields in the paper GUI: ON/OFF button and Send button. ON/OFF is needed so that writing on the menu item fields mistakenly won't alter the internal structures (keeping array of list of items) -- this is not so necessary since the data from the paper GUI won't be sent to the computer GUI unless you click on the Send button. However, we did this to make no extra fuss (make it clean). Also, the Send button sends all the data put in the field boxes (such as number of items orders and their names, tables number, order number) at once to the Sushiman and the Cashier. Lastly, we almost solved the synchronization issue of when multiple waitress takes order at the same time (writing on the paper and pressing "send" button at the same time). We are testing it for any bugs. We kept a linked list of each order depending on each order number to keep the orders separate -- we may use Java Hash Table later to make the progress quicker.
For Cashier's GUI (Statistics):
We are currently working on the Cashier's GUI for statistics: showing stats for most/least popular items depending on hourly, daily, monthly, seasonally, yearly preference, so that the owner of the sushi restaurants can easily decide on strategies to extract the best profit.
Method
This section describes in detail how we gathered feedback from potential users.
Participants
As in our previous assignments we decided to choose one smaller sushi restaurant and one larger sushi restaurant as our two main constituencies. We decided to interview a cashier from both restaurants. We would have liked to also interview a chef, but we were unable to find any sushi chefs with time. However, the cashiers seemed confident enough that they understood the chef's role and could evaluate any interface they may be using.
Apparatus
Our interviews consisted simply of bringing a laptop and running the interface. This simulated close to what it would be like when actually used: the cashier would be using the interface directly on a computer, and the chefs will be looking at it on a large touch screen. The setup ofcourse also consisted of our paper GUI and the Anoto pen system. We asked the cashier to pretend to order something using the pen and had her watch as the order was tabulated automatically on the screen.
Tasks
We decided to evaluate our interface using the same criteria that we established before during the contextual inquiry and task analysis.
Our application lends itself to three different categories of users: the waiters/waitresses, sushi makers, and the owner/cashier of the restaurant. Six tasks can be identified thus far which involve the waiters/waitresses, sushi makers, and the owner/cashier.
Easy:
(1) The first task our application requires is for the waiter/waitress to fill out a sushi order form on a customized sheet of digital paper. The waiter/waitress writes the quantity of each sushi and roll that customers want, identify the current table at which they are sitting (this information is needed when the waiter/waitress deliver the food or the bill to the customer), and check a ‘Send Order’ box indicating waiter/waitress completed to get customers’ order.
(2) A second task our application supports allows the owner/cashier to view the total amount of money each table owes. This function will require software to optically recognize the number the customer wrote indicating the number of sushi rolls they wanted and convert it into an integer. Each integer will be multiplied by the price of the given sushi roll and summed up to reveal the total amount spent.
Medium:
(3) The owner/cashier will need to be able to alter his or her menu and prices from time to time. This could be a result of supply and demand for the different types of sushi and also the cost of ingredients that go into food preparation. A separate interface should be provided to allow the owner/cashier to simply change his or her menu without worrying about the dynamics involved with the digital pen, paper, and optical recognition.
Hard:
(4) Information on order forms that have been completed by the waiter/waitress should appear on the computer screen in some kind of queue fashion which would allow the sushi maker to determine the next table in line to receive their food. Once the food was delivered, the order form would be removed from the queue and placed on a separate list for further processing, see task 5.
(5) Information regarding how much each given table owes remains on this list until the customer has paid and left. All orders would be grouped by table number for the possibility of a given table ordering multiple times during one sitting. Additionally, functionality should allow this information to be exported to a separate format which would be used to process a receipt. Also, a method to distinguish individual orders and printing separate checks is needed for the customers who want to have separate checks on the same table.
Procedure
Our procedure was quite simple. With each cashier we first gave them a tour of our interface. We showed them the available options, demonstrated how a new order could be placed, and spoke to them about additional functionalities that may be added. We then allowed them to experiment with the interface slightly. We felt that to truly measure the effectiveness of our given tasks, the user needed to atleast be somewhat familiar with our interface before being able to give a fair assessment. We then asked our cashiers to perform the above tasks.
Test Measures
When we came back to the users we were trying to measure a quantitative feeling of wether our interface improves the process of ordering food at a Sushi restaurant. We needed our test users to understand the potential in our product, and ultimately describe to us how our interface rates against the conventional ways of doing the same tasks. In regards to the tasks above, we asked our users to rate the tasks relative to the way they already perform them
To further quantify this we created a scale from 1-5:
1 - This task is much better performed through existing methods
2 - This task is slightly better performed through existing methods
3 - This task is on par with current methods
4 - This task is slightly better performed with the new system
5 - This task is greatly improved through the new system
Results
The results of our tests were very interesting. We'll break down results by each task and discuss comments that the users had. The tasks below refer to the tasks specified above and numbered 1-5.
Placing an order through our system
This task received a score of 3 and 4. The cashiers commented on the unreliability of the pen and ordering system, however they really liked how the order is instantly tabulated and sent to both interfaces instantly. A problem they said they experienced often was that when waitresses wrote down orders on slips, sometimes those slips would become unreadable to the chefs and they make the wrong order. They also mentioned that occasionally slips are lost as the kitchen is extremely hectic. They especially liked the feature that tabulates all the items to be made by item type so that the chefs can cook faster. They cited the unreliability of the pen and paper interface system as the reason they could not rate it higher.
Viewing Order Info
This task received two scores of 3. The cashiers liked our simple interface and said that the information was easily accessible, however the interface just didn't look as pretty and sophisticated as some of the software they already use. They particularly liked the fact that they did not have to enter in by hand the items that each table ordered, and would score the interface much higher if it had that level of sophistication.
Changing Pricing Info
This task requird a bit of wizard of oz setup becuase the interface was not completely finished for this. The cashiers we interviewed both gave this a score of 3. The reason being is that when they change prices, they still have to go through many of the same hassles. The physical paper menus need to be updated and reprinted. On their infrastructure side, it takes them about the same amount of time to change their prices on their exisiting systems as it would take them to change the prices in our new system.
Chef Interface Viewing
This task rated very highly with our system. We received a score of 4 and 5. The smaller sushi restaurant has a system in place conventional to many restaurants: the slips were hung up on a string infront of the chefs and as the chefs made one order, they would rip the slip off and then look at the next one. Our interface allowed them to quickly perform essentially the same action digitally but gave them the benefit of being able to see that they had to make 12 of X sushi roll for multiple orders.
"Check Please"
This task was similar to the first task in that it asked the cashiers to evaluate the interface they would be using. This task received a score of 4 and 4. The cashiers enjoyed the fact that they could simply look at an order and click a print button, and instantly they had a bill printed out. They say the potential in the system to reduce mistakes because the cashiers do not have to enter items for the order by hand. They said that this is really what gave this system an improvement over the current methods of taking orders.
Discussion
The pilot run was very helpful in that it provided some very useful specific feedback, as well as some general qualitative and overall feeling of the product type of feedback. One of the biggest hinderances to our product is also one of its great benefits: the Anoto pen system. With the current setup the pen just has too many errors and often times writing on our menu interface did not produce reliable results. The cashiers we interviewed expressed some frustration becuase since there are so many items on our menu, the boxes for each item are relatively small. If they had to make several markings to order an item, the box for that item quickly became full of ink and messy, and this made the pen work even worse. For our final product perhaps we will think about splitting up the menu into several menu sheets, or using larger menu sheets to order.
The other main issue that was brought to our attention through this usability study was that the interface is simple and achieves what it needs to do just fine, but it lacks the depth and sophistication of a professional product. The reality is that it will take us much longer to create an interface with that kind of complexity, but there are things that we can do relatively easily to try and get more of that look and feel. For our final iteration it would help to have the interface become a full screen or almost full screen setup. We also need to play around with color schemes and use graphics and beveled buttons.
Generally users were impressed with the project we created because the Anoto pen system was a totally different technology than what they currently use. It proved to be a double edged sword, but I think the customers realized that this system has a lot of potential to really streamline the sushi ordering process.
One specific thing that we want to improve is the custom ordering process. In the next iteration we plan to allow the waitress to add a "Special Instructions" sheet to any order. When the order is tabulated and sent to the system, these special instructions will be visible as an image to the chef. Items in an order containing special instructions will not be tabulated with all the other items that are made in bulk. Our test agreed that this was a good way to streamline the process for most orders which do not include special instructions, while allowing the flexibility of ordering whatever a customer wants to the customer.
Overall we asked our pilot users to really compare this interface and system with what they are currently using. The smaller sushi restaurant was using a paper based system, and calculating tabs and checks manually and then writing them out. The larger restaurant had a computer based system that still relied on a lot of physical human input. The smaller restaurant seemed more excited about the system we proposed, probably because they really had no automation yet. However the larger restaurant seemed hesitant to move away from the system they had already purchased. They were really looking for a system that contained the depth and complexity of a professional software suite. Despite this, they still had good things to say about our project.
Workload breakdown
Jae: Helped with pen and GUI integration. 20%
Sung: Interviewed the two cashiers. 20%
Anton: Lead GUI designer and programmer. 20%
Suneet: Helped with design and coding of interface. 20%
Johnathan: Designed paper UI and helped with coding. 20%
Appendices
Demo Script
Hello Mr./Ms._______, thank you for letting us come by and visit with you. Our names are <NAMES>. We really appreciate you for taking the time to speak with us. We are students at U.C. Berkeley and are taking a course in User Interface Design. We have observed the different tasks and points of concern related to your line of work with others in the past. Furthermore, we have a desire to aid you and others in your work by designing a system that would automate the sushi ordering process, and improve the efficiency of these kinds of restaurants. Based on what we have learned from your work, we have made considerable progress and have developed a prototype of the kind of system we have been talking about. We are interested in your thoughts and reactions on what we have designed so far. We are additionally interested in what feedback you can provide in improving our system. Before we move on we have a letter of confidentiality we would like you read through and sign. Basically it explains your rights and assures that we will not use this experiment to harm you in any way. [Let them read and sign the letter]. Now let’s move one to the experiment. What we are going to do is simply lay an interface out in front of you, and ask you to complete one of three tasks: act as a waiter/waitress and take an order; act as the Sushiman and fill the order; act as a cashier and process an order. We encourage you to speak your mind as you contemplate your objectives. In order to determine the effectiveness of our current design, we will be aiding you as little as possible. If you become totally confused and stuck, however, we will give you a nudge to help move you along. My partner and I will give a demonstration...
Consent Form
By signing this form I:
- understand there are no foreseeable risked involved with the experiment.
- understand the information obtained during the experiment will not be used to harm me in any way.
- understand my identity will remain confidential.
- understand the experiment will take approximately and hour.
- understand I am a volunteer and have the right to walk away from the experiment at any time without penalty.
- have the right to follow up on the progress of the research regarding the experiment.
Signature______________________________ Date____________________
