Proton++: A Customizable Declarative Multitouch Framework
Kenrick Kin, Björn Hartmann, Tony DeRose, Maneesh Agrawala
Abstract
Proton++ is a declarative multitouch framework that allows developers to describe multitouch gestures as regular expressions of touch event symbols. It builds on the Proton framework by allowing developers to incorporate custom touch attributes directly into the gesture description. These custom attributes increase the expressivity of the gestures, while preserving the benefits of Proton: automatic gesture matching, static analysis of conflict detection, and graphical gesture creation. We demonstrate Proton++’s flexibility with several examples: a direction attribute for describing trajectory, a pinch attribute for detecting when touches move towards one another, a touch area attribute for simulating pressure, an orientation attribute for selecting menu items, and a screen location attribute for simulating hand ID. We also use screen location to simulate user ID and enable simultaneous recognition of gestures by multiple users. In addition, we show how to incorporate timing into Proton++ gestures by reporting touch events at a regular time interval. Finally, we present a user study that suggests that users are roughly four times faster at interpreting gestures written using Proton++ than those written in procedural event-handling code commonly used today.
In Proton++ the developer can specify a custom touch direction attribute. (a) The direction is computed by taking the vector formed by the last two positions of the touch and (b) binning it to one of the four cardinal directions. Combining the hit-target and direction attributes, the developer can specify a gesture to translate a star (denoted as 's') with varying degrees of specificity: (c) north only, (d) north and south only, (e) in any direction.