From CS160: User Interface Design Sp12

Contents 1 Members 2 Ideas 2.1 Idea Sketches 3 Selected Idea 4 Project Description 4.1 Target User Group 4.2 Problem Description & Context 4.3 Why is a Kinect application a good solution for the problem?

Members

• Praneet Wadge
• Tobit James Co Narciso
• Varad Kishore
• Tamzid Islam

Ideas

1. osu!Kinect Kinect interface to the osu! rhythm game. Sample video of PC version (controlled by mouse, keyboard or tablet)
2. ASL Translator Uses the Kinect to translate American Sign Language for the people who are untrained with it to communicate with the deaf/mute.
3. Interactive Street Ads A platform for large-display advertising that interacts with users (maybe through games), talks and allows options as users walk by.
4. Security System Having the Kinect mounted on the gate to discern whether users are walking past, or lurking and aren’t recognized through facial recognition.
5. Gesture Based Authentication Uses gestures and sounds to authenticate a person, as a security system for safes, etc.
6. Parking Finder app for garages so drivers have immediate knowledge of open spots.
7. Input-system for paraplegics Enables users to navigate menus through neck movements.
8. Kinect assisted vaccum cleaning Use motion tracking using the Kinect’s RGB camera and depth sensor to coordinate Roombas to vacuum a room.
9. Martial arts training The Kinect can teach martial arts by having the user follow a series of movements, tracking them, and providing suggestions.
10. Elegance training/etiquette Similar to 9. but with posture, etiquette, etc.
11. Screenless Music browser allows user to remotely change the track of his media player even without any screen-like interface.
12. Kinect in the Shower Music browser that can be controlled while in the shower, (kinect mounted on top of showerhead)
13. Interactive Whiteboard An interactive whiteboard solution with a Kinect and a projector such that the user can select and drag around items on the board.
14. Automated screencasting Kinect tracks the professor’s movements and pans to keep camera focused on the professor, no matter where he may be on the stage.
15. Card Game Referee Uses the Kinect’s camera to referee (trading) card games.
16. Driver activity detector Senses if the driver of the car is nodding off, reaching for something, not paying attention to the road, or other dangerous behavior and sounds an alarm to alert (wake up?) the driver.
17. Message Writing and Forwarding Uses the Kinect to detect a user’s writing motions and converts it into text which can be sent to other people with a gesture.
18. Combine WiiRemote w/Kinect for FPS Using the Kinect’s skeletal tracking to implement dodging and docking, while using the Wiimote for button input, and pointing capabilities to create a first-person shooter that involves actual body movements.
19. Drag&Drop File Transfer Transfer files along the network using an interface where people can literally pick up and drop the objects to be transfered.
20. Human recognition through motion Kinect collects data on movement and human behaviour, body rhythm, patterns and uses ML to detect whether the person in the room is authorized
21. 3D Desktop A 3D UI for the desktop where the user can reach for items with their hands and the program mimicks the movements in 3D space with an animated arm.
22. Scratch + Kinect Interface Scratch (used in CS10) with Kinect gesture recognition to teach kids how to program in a more dynamic manner.
23. 3D Virtual Girlfriend (LovePlus+Kinect) - Using the Kinect, gives the illusion of physically interacting with a virtual girlfriend (hugging, kissing, talking, etc. girlfriend can also know user’s location), targeted towards socially inept consumers.
24. Toddler NUI games Training toddlers to learn shapes, letters, via gestures and movements (e.g. put square peg in square hole type exercises).
25. Cooking Instructor Cooking instruction/trainer for new college going cooks, who have no idea how to mix things, beat eggs etc.
26. Voice tracker the camera finds and focuses on the person currently talking in a conference, when recording a conference.
27. 3D Medicine Database Interface Allows the user to sort through different categories of drugs with controls in three dimensions (e.g. up/down for purpose of drug (antihistamines, etc.), left/right for family of drugs, forward/back for the brand)
28. Pharmacy checkout system Kinect tracks a customer and the customer can point to an item to add to cart or view information, when customer approaches checkout counter, the system automatically displays that customer’s cart to the attendant.
29. Virtual Receptionist (Concierge/Librarian/Tour Guide) Interacts with and tracks/follows the user face to face, including eye movements to give directions/information.
30. Picture Scroller Hands free scroller for painters, artists, others who get their hands dirty, but then want to look at spec sheets, design requirements etc.
31. Linear Algebra + Kinect Interactive linear algebra for students and professors (create matrices, add numbers, visually push them together to dot product etc). Also draw the vectors in 3D using the Kinect's depth sensor.
32. Visualisation for DJs Creates a projected visualization (like in Windows Media Player) and an interface for DJ/conductors for mixing and matching music tracks which responds to the artist’s movements and can be seen by the audience
33. Music Maker A simple interface for non-musicians to create music where specific gestures indicate hitting certain notes.
34. Song Visualizations Similar to #32, but targetted at users as a musical game to create ‘Electric Sheep’ style visualization for songs through movements, hence creating trippy unique artwork.
35. Cricket Referee Hand gesture controlled score board for cricket referees.
36. O-Chem Modeller Organic chemistry education where you can visually see molecules and put them together to understand chemical reactions, balance equations, remove atoms etc to change reactants etc.
37. Protein/RNA Modeller Protein/RNA modelling application for researchers who can physically manipulate 3D structures of proteins or RNA to observe their folding, separating their components, etc.
38. Coffee bar Customizable coffee bar where users can go build their own coffee, and the Kinect is interfaced with the coffee machines to add how much ingredients the customer wants.
39. Home Efficiency Central home efficiency interface controlling ACs, windows, appliances etc for home owners looking to save money, turn off electricity when not in use etc.
40. Integrated Remote Whole house remote where users can sit on the sofa, turn on the TV, radio, etc while all sitting in one spot.
41. Visual website builder where you physical manipulate buttons, links etc into an interface, and the code is generated on the backend.
42. Choreography aid for directors to build and perfect fight scenes.
43. Plugin for web apps enables use kinect on websites for interactive experiences for users.
44. Word Dodgeball Teaches young children vocabulary, sentence construction, etc.by having them catch certain words and avoid incorrect ones. Can also be used for math equations.
45. Trigonometry/Geometry Aid Helps middle school/high school students by letting them build and manipulate angles or polygons, polyhedra, or other 2D/ 3D objects with via hand movements or other gestures.
46. Drum machine /percussion that teaches tones, drumming patterns to new musicians, or allow musicians to drums.
47. Visual electric circuit builder to teach children basic electrical engineering (how to connect bulbs in parallel and series etc).
48. Rube Goldberg machine simulator to teach children basic physics, and mechanical engineering.
49. First aid learning tool where the kinect is mounted on the ceiling and watches you as you check pulse, broken bones etc on a lying down subject.
50. Blind vision a system that can be mounted on a visually impaired person and will help him move by detecting objects, humans, or animals in front of him. We can use sound or haptic feedback.
51. Bathroom guidance utility An utility that helps visually impaired people to move around in a bathroom or other similar locations, without help of another person, by giving them directions to get from one point in the room to another. For example, the system would detect available stalls in the bathroom and check if anyone needs help with directions, it would say ‘stall 1 available, turn left.’
52. Virtual sculpting Use the Kinect to sculpt a blob of virtual clay on the screen, as a form of rapid 3D modelling/prototyping (applications like Rhinoceros have large learning curves).
53. Productivity mini-games with a kinect (juggling etc), that lets people get out of their seats to relieve stress, avoid long hours staring at a computer screen.
54. Mind mapping software for brainstorming Useful for designers, people working on projects etc. You say an idea out loud, it types up and generates a bubble, which you can manipulate then with gestures across a screen to connect with other bubbles etc, shows related suggestions relating to your input from around the web, in lighter bubbles
55. Motion-sound Synesthesia Renders any user motion as sound. Idea is to produce music by moving in random motions (dancing?) rather than to produce a desired melody as in #33 (In a way, it's converse). Similar to #34 but produces sounds.

Idea Sketches

Interactive Ad Game idea (#3)

Shower implementation idea (#12)

Mind Mapping (#54)

Driver Activity monitoring (#16)

Sound Visualization ideas #32 and #34

Wiimote+kinect Shooter Game (#18)

Linear Algebra tool (#31)

DJ Mixer (#32)

First aid tool (#49)

Interactive Whiteboard (#13)

Selected Idea

The idea we ended up choosing was #3 Interactive Advertisements. We chose it after one by one eliminating projects that would have required greater graphics knowledge than could be learnt in the breath of this course. We also eliminated ideas that were simply adaptations of existing touch technologies with a Kinect twist, and games as well. We were still left with a number of 'cool' ideas, but the parameters of the project narrowed it down to interactive advertisements (#3) and the mind-mapping software (#54).

Project Description

Interactive (Street) Advertisement Platform

Target User Group

Our idea is to create an advertising platform that produces ads that people can interact with through the Kinect. Thus, Ad agency groups are our customers, in that they are the people we should be pitching this idea to. However, the 'users' of the interactive ad interface will primarily be everyday shoppers going to malls and stores, or pedestrians on the street, who may be interested in looking at ads. Ads are tailored to specific audiences, and thus our target user group will change as the ads change.

Problem Description & Context

Regular ads are not very memorable and hardly interactive unless they are extremely well designed or shocking/distinctive. The ratio between how much an agency/company spends on advertising and how many customers actually come and buy a product is always under contention and can determine the life or death of the company. More physical interaction with a company’s advertisements not only entices a customer, but can generate a sense of personal attention and positive feeling in them.

Why is a Kinect application a good solution for the problem?

To increase interest for the user and raise impressions for the ad agencies, we wish to create ads that passers-by can interact with in interesting ways. Examples of these interactions include advertising "games" where the user can play a game to win coupons or other prizes, a "pitchman" who follows and talks to the user as he moves around, having the user's comments appear on screen as they voice them out, etc. In each of these, the Kinect's motion and audio tracking can be used to great effect, to create a more interactive ad. An advantage of using the Kinect over a touch-based interface is that in a public setting, this can cause hygienic problems as many people would be touching these interfaces. Using the Kinect reduces this problem as there would be no need for users to touch anything as they interact with the ad.

Brainstorming additional ideas for "Interactive Street Ads" (#3, Our chosen project)

Interactive Ad Game idea (#3)

In creating the platform for ad agencies we would have to program the infrastructure of certain types of interactions that the ad agencies can leverage by simply uploading a coupon, a blue-screen video or some meta-data. We started brainstorming with the following:

• Catch a coupon - the user would be able to play a quick trivial game to win a coupon. The coupon would be displayed as a QR code for the user to collect using a smartphone, or simply a coupon number the user could jot down.
• Following Pitchman - a blue screen video of a salesman would follow the user as he walked across the screen, trying to get the user's attention to make an offer or talk about a product.
• An interactive model - for instance a car, the user would be able to view it in 360 degrees.
• A bot style conservation and comment roll, etc.

---